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received the 8S. B. degree from the Massachusetts 
Institute of Technology, Cambridge, in 1944, and 
the degrees of M. A., M.E.S., and Ph.D. in 1948, 
1949, and 1950, respectively, from Harvard Uni- 
versity, Cambridge, Mass. He served with the U.S. 
Navy as a radio technician from 1944 to 1946. 

From 1950 to 1953, Dr. Elias was a Junior Fellow 
in the Society of Fellows at Harvard University 
doing research on information theory. He became 
an Assistant Professor at M.1.T. in 1953, in the 
Electrical Engineering Department and Research 


Laboratory of Electronics. In 1956, he became an 
Associate Professor. He has been working on two- 
dimensional random processes and pictures as in- 
formation sources, efficient coding of redundant 
sources, and noisy channel coding problems. 

Dr. Elias is a member of Sigma Xi, Eta Kappa 
Nu and the Institute of Mathematical Statistics. 
He was chairman of the 1956 M.I.T.-PGIT Informa- 
tion Theory Symposium, and currently is chairman 
of the IRE Technical Committee on Information 
Theory and Modulation Systems. He was recently 
elected a member of the PGIT Administrative 
Committee. 


‘Two Famous Papers 


PETER ELIAS 


It is common in editorials to discuss matters of 
general policy and not specific research. But the 
two papers I would like to describe have been 
written so often, by so many different authors 
under so many different titles, that they have earned 
editorial consideration. 

The first paper has the generic title “Information 
Theory, Photosynthesis and Religion” (title cour- 
tesy of D. A. Huffman), and is written by an engi- 
neer or physicist. It discusses the surprisingly close 
relationship between the vocabulary and concep- 
tual framework of information theory and that of 
psychology (or genetics, or linguistics, or psychiatry, 
or business organization). It is pointed out that the 
concepts of structure, pattern, entropy, noise, trans- 
mitter, receiver, and code are (when properly in- 
terpreted) central to both. Having placed the 
discipline of psychology for the first time on a 
sound seientific base, the author modestly leaves 
the filling in of the outline to the psychologists. He 
has, of course, read up on the field in preparation for 
writing the paper, and has a firm grasp of the 
essentials, but he has been anxious not to clutter 
his mind with such details as the state of knowledge 
in the field, what the central problems are, how they 
are being attacked, et cetera, et cetera, et cetera. 

There is a constructive alternative for the author 
of this paper. If he is willing to give up larceny for 
a life of honest toil, he can find a competent psychol- 
ogist and spend several years at intensive mutual 
education, leading to productive joint research. But 
this has some disadvantages from his point of view. 
First, psychology would not be placed on a sound 
scientific base for several extra years. Second, he 
might find himself, as so many have, diverted from 
the broader questions, wasting his time on problems 
whose only merit is that they are vitally important, 
unsolved, and in need of interdisciplinary effort. 
In fact, he might spend so much time solving such 


problems that psychology never would be placed on 
a sound scientific base. 

The second paper is typically called ‘he Opti- 
mum Linear Mean Square Filter for Separating 
Sinusoidally Modulated Triangular Signals from 
Randomly Sampled Stationary Gaussian Noise, 
with Applications to a Problem in Radar.” The 
details vary from version to version, but the initial 
physical problem has as its major interest its obvious 
nonlinearity. An effective discussion of this problem 
would require some really new thinking of a difficult 
sort, so the author quickly substitutes-an-unrelated. 
linear problem which is more amenable to analysis. 
He treats this irrelevant linear problem in a very 
general way, and by a triumph of analytical tech- 
nique is able to present its solution, not quite in 
closed form, but as the solution to an integral equa- 
tion whose kernal is the solution to another, bi- 
variate integral equation. He notes that the problem 
is now in a form in which standard numerical analysis 
techniques, and one of the micromicrosecond com- 
puters which people are now beginning to discuss, 
can provide detailed answers to specific questions. 
Many authors might rest here (in fact many do), 
but ours wants real insight into the character of the 
results. By carefully taking limits and investigating 
asymptotic behavior he succeeds in showing that in 
a few very special cases (which include all those 
which have any conceivable application or offer any 
significant insight) the results of this analysis agree 
with the results of the Wiener-Lee-Zadeh-Raggaz- 
zini theory—the very results, indeed, which Wiener, 
Lee, Zadeh, and Raggazzini obtained years before. 

These two papers have been written—and even 
published—often enough by now. 

I suggest that we stop writing them, and release 
a large supply of manpower to work on the exciting 
and important problems which need _ investiga- 
tion. 
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An Experimental Investigation of Some Properties of 
Band-Pass Limited Gaussian Noise’ 


KJELL BLOTEKJARt 


Summary—tThe probability distribution of time intervals between 
successive zero crossings of band-pass limited Gaussian noise is 
determined experimentally for a number of different filters having 
nearly rectangular frequency characteristics. For one particular 
filter the distribution of time intervals between crossings of levels 
different from zero is also found. 


INTRODUCTION 


HE DISTRIBUTION of time intervals between 
ale successive crossings of any level, and more par- 

ticularly the zero level, of band-pass limited 
Gaussian noise cannot be derived explicitly by analytical 
methods, and numerical evaluation tends to become 
very involved. 

The purpose of the present investigation is to determine 
this distribution by experiment, to study its relation to 
the noise power spectrum, and to discuss the results in 
view of the solution of a related problem which can be 
solved analytically. 

Rice’ presents a probability density function of time 
intervals between successive zero crossings obtained ex- 
perimentally by M. E. Campbell from a limited amount 
of data. The present work is based on a different measur- 
ing technique, and supplements and extends Campbell’s 
results. 


PRINCIPLES OF MEASURING TECHNIQUE 


The noise signal analyzed was derived from an ordinary 
video amplifier with a flat noise power spectrum; the 
desired noise spectrum was obtained by combining this 
noise generator with a low-pass and a high-pass filter 
having variable frequency limits. 

The analysis was carried out directly on the noise 
signal. A Schmitt trigger was used to convert the noise 
into a square wave, see Fig. 1(b). The zero crossings of this 
square wave coincide with the zeros of the noise wave- 
form. These square waves were again converted into a 
sawtooth wave, shown in Fig. l(c). The slopes of the 
sawtooth pulses were maintained constant, the height 
of these pulses then being proportional to the time interval 
between successive crossings of the zero level. By applying 
the sawtooth waveform to another Schmitt trigger and 
an electronic counter, it is possible to count the number 


* Manuscript received by the PGIT, September 30, 1957. 

+ Norwegian Defense Research Establishment, Kjeller, Norway. 
This work was carried out as partial fulfillment of the final examina- 
tion at the Norwegian Institute of Technology. 

18. O. Rice, “Mathematical analysis of random noise,” Bell Sys. 
Tech. J., vol. 23, p. 282; July 1944, and vol. 24, p. 46; January 
1945. 
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of time intervals exceeding a certain preset value. Inj 
this manner the integrated probability distribution was) 
derived, and hence, by straightforward manipulations,, 
the desired probability density function is obtained. | 


Discussion oF HrRRors [INVOLVED IN THE 
MraASURING TECHNIQUE 


Two main sources of error are involved: statistical’ 
fluctuations, and systematic errors due to imperfections 
in the measuring equipment. 1 

Purely statistical fluctuations can be made as small | 
as desired by increasing the number of observations JN, { 

e., the number of counts, because the relative fluctua-| 
tions are inversely proportional to the square root of NV | 

The remaining limitation on the accuracy of the} 
measurements is due to imperfections in the experimental | 
technique described above. A thorough analysis of the | 
measuring technique revealed that the relative un-}| 
certainty was between 1 and 2 per cent. It was necessary | 
therefore to choose the number of counts sufficiently large 
to make the statistical fluctuations smaller than this per- | 
centage. 


(a) 


Go) 


: ba 


(d) 


Fig. Ea gue of measuring technique. (a) Original noise signal. 
(b) Square wave derived from (a). (¢) Sawtooth pulses derived 
from (b). (d) Square pulses indicating that the sawtooth pulses 
(c) exceed a preselected level. 


RESULTS 


The distribution of time intervals between successive 
zero crossings was determined for a number of different 
filters. Each curve was found from approximately 30 
measured points. 


| 958 


_In order to compare the experimental results with 
theory, it is desirable to use filters having frequency 
haracteristics which can be expressed in a simple math- 
matical form. In the present investigation it was aimed 
t obtaiming rectangular frequency characteristics. The 
ictual slopes of the edges were approximately 200 db per 
pctave, while the intended flat portion of the filter 
tharacteristics had irregular variations of maximally 
?.5 db. Thus, exact mathematical expressions for the 
oise power spectrum could not be obtained, and the 
[ston of rectangular spectra is a relatively crude 
pproximation. This must be borne in mind when theo- 
‘etical and experimental results are compared. 

_ At low frequencies the power spectrum of the noise 
jource is no longer flat due to various reasons (flicker 
tffect, etc.). Accordingly, it was found necessary to elimi- 
hate this part of the spectrum by filtering. Therefore, 
hoise with a power spectrum starting at zero frequency 
tould not be examined, but to obtain a reasonable approxi- 
mation the lower frequency limit of the filter was chosen 
hs low as permissible, at 440 cps. The higher frequency 
junit was 10, 250 cps, which was well below the maximum 
frequency that could be analyzed by the measuring 
bquipment. 


PROBABILITY DENSITY p(?) 
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periodic component of frequency fy would be a 6 func- 
tion at g¢ = a. When the lower frequencies are added, 
the distribution becomes broader, and the maximum 
moves towards longer time intervals. The second maxi- 
mum at ¢ & Y is due to the case where the noise signal 
has two maxima and one minimum between successive 
zero Crossings. 

The dotted curve in Fig. 2 represents a probability 
density function which was calculated by Rice. It gives 
the probability density of a noise signal /(¢) crossing the 
zero level with a negative slope at time ¢t + 7, given that 
I(t) is crossing the zero level at the time ¢ with positive 
slope. 

Apart from time intervals between sucessive zero cross- 
ings, this distribution also includes time intervals with 
intermittent zero crossings. Therefore, for all values of ¢ 
this distribution should be situated above the experimental 
curve. Rice has shown, using theoretical arguments, that 
the two curves should practically coincide for values of ¢ 
less than three. Fig. 2 shows that this is in fact the case. 

For somewhat higher values of ¢, however, the experi- 
mentally obtained distribution exceeds the theoretical dis- 
tribution found by Rice. This might be due partly to the 
low-frequency cutoff in the noise spectrum, and partly to 


| The distribution of time intervals between successive 
vero crossings for noise passed through this filter is shown 
i Fig. 2, where the abscissa is expressed in terms of 
b = 2nfor, where fy is the upper frequency limit of the 
Hilter. In transforming from 7 to ¢ all distributions p(¢) 
ecome equal, no matter the value of fo, provided the 
shape of the power spectrum is the same. This distribu- 
tion has a distinct maximum at ¢ & 4 and a less distinct 
aximum at ¢ & 9. The physical reasons for these maxima 
are as follows. The corresponding distribution for a 


10 15 20 


C2 2 Uf li WE 


Fig. 2—Distribution of time intervals between successive zero crossings. Filter: 
approximately low pass. Related theoretical curve shown by dotted line. 


the irregular variations over the pass band of the filter 
characteristic. 

Fig. 3 gives the results of measurements with several 
filters with upper frequency limits at 10,250 cps, and 
with variable lower frequency limits. In this figure it 
should be noted that the vertical scale is enlarged for 
values of ¢ larger than 6.5. 

As the lower frequency limit is increased, rendering 
the noise spectrum narrower, the distributions become 
steeper round a single maximum which is moving towards 
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Fig. 8—Distribution of time intervals between successive zero 
crossings. Filters: fixed upper frequency limit, variable lower 
frequency limit. 


shorter time intervals (7.e., smaller g). This is in agree- 
ment with the fact that the distribution for infinitely 
narrow filters should tend towards a 6 function at ¢ = 7. 

Fig. 4 gives the distributions of time intervals between 
successive crossings of levels other than zero, for a filter 
with frequency limits 440 eps and 10,250 eps. All distri- 
butions are normalized to give the same area. The levels 
examined were all positive, and between the crossings the 
signal was above the level. 
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Fig. 4—Distribution of time intervals between successive crossings 
of some different levels. Filter: approximately low pass. 


CONCLUSION 


| 


: 
! 


The main difficulty in comparing the experimental | 


results with the related theoretical results of Rice appears 


to be the lack of ideal filter characteristics, and not the | 


lack of accuracy in the experimental method. Further 


work to test the theory should therefore concentrate on | 
producing filter characteristics which can be represented — 


by simple analytical expressions. 
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Summary—The method of construction of lossless symbol coding 
atrices for one-error correction is illustrated for the case when the 
rime symbol order is three, and the application of this matrix to 
he penny-weighing problem is described. This method is then ex- 
tended to those cases in which the symbol order is 22, 23, 24, 2°, 
2, 38, 34, 35, 5%, 55, 54, 72, 73, and p?, where p is any higher prime. 
his extension is based on the concept of the master iterating 
atrix. These matrices are given for the first thirteen cases cited, 
nd their existence is demonstrated for p?. 

This paper concludes with a short description of Zaremba’s 
ondition, and its application to various problems, and more par- 
ticularly to the hypothetical one-error correcting close-packed code 
ith the symbol order 6. 


INTRODUCTION 


N a former note,’ the writer described the construc- 
tion of lossless symbol coding” matrices for one-error 
correction when the symbol order is prime, and two 

dditional singular cases of close-packed coding for two and 

hree-error correction, respectively. 

An elaboration of the one-error correcting binary code 

jvas published in an industrial journal,’? but the only 

additional error correcting lossless codes (with no re- 
striction placed on the individual symbol errors) published 
since 1949 are due to Zaremba,* who showed on group- 

‘theoretical grounds that a close-packed code book for 

message coding’ always exists for one-error correction, 

ens the symbol order is a power of a prime. The essential 


ortion of these notes will be devoted to those cases of 
lossless nonprime symbol coding for which coding matrices 
can be constructed systematically. 
In the first part of the discussion, a recapitulation is 
iven of the matrix construction formerly described,’ for 
lossless one-error correction coding when the symbol 
order is three, and the application of this particular code 
to the penny-weighing problem is described. 
In the second part, it is shown how the construction 


_ * Manuscript received by the PGIT, February 25, 1938; revised 
manuscript received June 15, 1958. 

+ Phileo Corporation, Philadelphia, Pa. 
1M. J. E. Golay, “Notes on digital coding,’’ Proc. IRE, vol. 
37, p. 657; June, 1949. : 
_ 2The expressions ‘symbol coding, message coding, cor- 
rector,” “characteristic,” etc., are given the same meaning as in a 
former publication (M. J. E. Golay, “Binary coding,’ IRE Trans. 
on INFORMATION THEORY, no. PGIT-4, pp. 23-28; September, 
1954). A “losslesssymbol coding matrix” yields a ‘‘close-packed code,”’ 
but cases are conceivable—although unknown—in which a close- 
packed code exists but symbol coding is impossible. 

3R. W. Hamming, ‘Error detecting and correcting codes,” Bell 
Sys. Tech. J., vol. 29, pp. 147-161; April, 1950. Mention should be 
made of an earlier article by Shannon, in which the “somewhat arti- 
ficial’ case of coding seven-bit words against one error by means of 
three parity checks is described and attributed to Hamming (C. E. 
Shannon, ‘‘A mathematical theory of communication,” Bell Sys. 
Tech. J., vol. 27, p. 418; July, 1948.) 

48. K. Zaremba, ‘Covering problems concerning abelian groups,”’ 
J. Lond. Math. Soc., vol. 27, pp. 242-246; April, 1952. 
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Notes on the Penny-Weighing Problem, Lossless 
Symbol Coding with Nonprimes, Etc.” 


MARCEL J. E. GOLAY+ 


of lossless symbol coding matrices may be extended from 
code symbols of prime orders to code symbols of orders 
2, 2°; 2, 29,3388 78, MD oy ah walle anyaconane 
of an odd prime number. 

The third part of the discussion contains observations 
on various solved or unsolved coding problems. 


Ture Penny-WEIGHING PROBLEM 


Given a balance which can be used to determine 
whether two weights are equal, or which is heavier, and 
given a certain quantity of pennies, of which at most one 
may be heavier or lighter than the standard weight, it is 
asked to determine what paired assemblies of pennies 
should have their weight compared with each other, in 
order to find, with a minimum number of operations, 
which penny, if any, is too heavy or too light. It is also 
required that the weighing program be completely pre- 
determined, and thus be not affected by the results of the 
successive weighings. 

Each penny may be in one of three possible states, too 
light, of correct weight, and too heavy, and its state is 
thus expressible by a ternary symbol. The information 
gathered from each weighing is also expressible by a 
ternary symbol. These circumstances suggest an analogy 
with the transmission and reception of messages composed 
of ternary symbols, of which one at most may be received 
in error. It may be surmised that, for instance, three 
weighings, yielding log,27 information bits, should 
determine which of 13 pennies, if any, is too light or 
too heavy. A form of lossless coding appears required to 
solve this problem, since each penny may be off-standard 
in two ways, thus yielding 26 possibilities, to which 
must be added the twenty-seventh possibility of all being 
of standard weight. 

Consider now the coding matrices described in the 
former publication,’ for the case p = 3. The simplest 
matrix, corresponding to n = 1, covers the trivial case 
of no information message and a single transmitted 
ternary check symbol, X,. The coding matrix for this 
case consists of a single “‘1”’: 


Xy 


(1) 


= 


and since there are no information symbols, the trans- 
mitted check symbol will be zero. 

The passage from n = 1 to n = 2 is made by writing 
the single term matrix (1) three times, writing below the 


numbers 2, 1 and 0, and adding a ° column at the end: 
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Vana) ay 
| | aeen() (2) 
2 O15 eet 
The passage from n = 2 to n = 3 is accomplished 
similarly: 
Vie ViVi s¥ oc Voped oe Yon) oo ee ee 
—— = —— = - ! = 
(er aK a i 
) i ere leet) eal i : LO eee) (3) 
TOTO Le Ok eee 1h ene ee) 
{ 
Di Dh 2 OPE vile clan ile OTL ee Ome mae 
and so forth. 
The coding equation:' 
- 
En = AG, —- SS Amk Y, = 0 (mod. p) (4) 


n 


k = 1, NV ae 


pn: is ob. <All nee aa 


is used to calculate the XY check symbols at the trans- 
mitting end, and to calculate the corrector H,, at the 
receiving end; the terms of matrix (3) are the coefficients 
of the Y’s and X’s in (4), for the case n = 3, p = 3. 

The 13 ternary numbers written vertically in the 
13 columns of (3), which are termed the “base character- 
istics’”’ of the X’s or Y’s, and the other 13 numbers obtained 
by multiplying the individual digits of the base character- 
istics by 2 modulo 3, which are termed the ‘derived 
characteristics” of the X’s or Y’s, are all different and 
represent the two ways in which one of the 13 X’s or 
Y’s could have been received in error. Inspection of the 

0 
corrector, when it is not 0, indicates which symbol was 
0 
received in error, and whether the error was +1 (base 
characteristic) or +2 (derived characteristic). 

The one-to-one correspondence between any nonzero 
corrector, and the base or derived characteristic of one of 
the 13 message symbols constitutes the lossless property 
of this one-error correction code, and it will be readily 
seen that the iteration process described above conserves 
this property when passing from n ton + 1. 

Consider now that the X’s and Y’s represent 13 pennies, 
all equal when shipped, but one of which may have had 
its weight altered upward or downward before being 
received. This corresponds to the case of a known trans- 
mitted message which is not delivered as such to the 
addressee. Instead, the addressee is given the /,,’s obtained 
from the three weighing operations, and asked which 
penny, if any, had its weight increased or decreased. 

Inspection of the coding matrix indicates a difficulty 
in determining the L,,’s by weighing various assemblies 
of pennies: none of the matrix lines contains 1’s and 2’s 
in equal number. On the other hand, if an extra penny of 
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correct weight is available—a ‘“catalyzer” conveying no 
information as such—it can be placed on one side of the 
scale together with Y,, Y., Y; and Y4, while Ys, Yo, Yz, 
Y, and X; are placed on the other. The result of this 
weighing operation clearly yields the addition modulo 3 
required to determine /;. 

The weighing operations required to determine /, and 
E, would require three extra and nine extra (good) 
pennies, respectively, but we may obtain instead the sums 
EE, and E,E;. When the elements of #, and ££; are 
added modulo 3, the following 13 numbers are obtained 
under Y, - X3: 1020021221011. Likewise, the H,F, 
numbers are: 0002222111101. This indicates that the 
determination of H,H; and H,H#; can be effected by 
weighing operations in which, as for EH, five received 
pennies are weighed against four other received pennies 
and the extra (good) penny. These three weighing opera- 
tions determine completely which penny, if any, is too 
light or too heavy. 


LOSSLESS SYMBOL CoDING WITH 
Powers oF PRIMES 


When the order of the message symbols is a power of 
a prime, p*, n check symbols should cover p*” — 1 possi- 
bilities of transmission errors. 

Any received symbol may be in error in p* — 1 ways, 
and if at most one symbol per message may be received 
in error, a first condition for lossless coding is that the 
total number of information and check symbols in a 
message be 


The coding equation for such a case has the general 
form: 


Eni = Xnc + Do atiY,, =0 (mod. p) (5) 


i esse eS a5 iad ape a 


Pies 


where Xni°°* Xmq and Y,, --+ Y,, are the q elements of 
the X,, symbol and Y, symbol, respectively: 


D. = CER ey re Xs.) 
We: = Che ies Vie Ne) (5a) 
E,, ae Hints Loess gy Y Dee 


The matrix formed by the coefficients of the Y’s and 
X’s in (5) will be termed the (n, p, q) matrix. When 
p = q = 2, the trivial case of a message containing no 
information and a single check symbol is represented by — 
the (1, 2, 2) matrix: 


Gr XG 

| 

os ee (6) 
| 

<u Oe wel 


| 
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The passage from the (1, 2, 2) matrix to the (2, 2, 2) 
matrix is analogous to that described earlier and in the 
ormer publication’ for the cases g = 1. The matrix (6) 
is first written four times horizontally and the four 
“iteration matrices” 


Le al 
ee0s 


Pee ee Oot 
Cet 7 et 


ae ey (7) 


0 0 


are written below the four identical matrices first written. 
The (2, 2, 2) matrix is completed by a double column 
(consisting of an all-zero matrix above a (1, 2, 2) matrix. 


Ys; Yi2 Yo ee Ys Y 39 X14 X12 X 91 X 99 


H 


il 0 il 0) 1 QO il 0) 0 O 
: (8) 
0) il 0) i 0) il 0 il 0 0 


eect Oa OS roy =o. 0 


| 


i Oe) mei Og 0 Opes 


It can be verified by inspection of matrix (8) that all 
15 possible nonzero correctors are reproduced by the 
10 columns of (8), which are the 10 base characteristics 
of the 5 pairs of Y’s and X’s, and by the 5 derived 
(characteristics formed by the sums of the two base 
scharacteristics of a Y,,:Y,. pair or of an X,,;Xno pair. 
‘Thus, all 15 possible cases of an error in one of 5 trans- 
mitted biquadratic symbols are covered exactly once, 
-and the code calculable with (8) is close packed. 

When passing from biquadratic to quaternary symbols, 
ithe following convention will be used: 


00-0 O1-1 10-2 11-3 (9) 


‘With this convention, the 64 messages belonging to the 
(2, 2, 2) code, and determined by giving the Y symbols 
all combinations of numerical values and calculating the 
_X’s with matrix (8), can be written compactly: 


00000 02022 10012 12030 20023 22001 30031 32013 


(00113 02131 10101 12123 20130 22112 30122 32100 


00221 02203 10233 12211 20202 22220 30210 32232 
00332 02310 10320 12302 20311 22333 30303 32321 (10) 
o1011 03033 11003 13021 21032 23010 31020 33002 
(01102 03120 11110 13132 21121 23103 31133 33111 
01230 03212 11222 13200 21213 23231 31201 33223 


01323 03301 11331 13313 21300 23322 31312 33330. 


_ When passing from the (2, 2, 2) matrix to the (8, 2, 2) 
matrix, the (2, 2, 2) matrix is written four times hori- 
zontally and a third double line is added, consisting of 
the four iteration matrices, repeated each five times. 
A twenty-first double column, consisting of two zero 
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matrices above a (1, 2, 2) matrix completes the (3, 2, 2) 
matrix. 

It has been observed that the 15 nonzero correctors 
associated with the (2, 2, 2) matrix are reproduced by 
the single columns of the (2, 2, 2) matrix or by the sums 
of column pairs. A set of three further observations will 
now be made, namely: that the first column of the four 
iteration matrices are all different from each other 


é es and °', that the second column of all iteration 


ees 

LOE 0 

ioe o 
and that the sums of the column pairs of the four iteration 

Urals i 0 
aoe O 
Thus any base or derived characteristic of the (2, 2, 2) 
matrix is continued, in the (8, 2, 2) matrix, by one of the 
- * ; or A Therefore, it is 
concluded from these four observations that the 20 first 
pairs of X and Y columns of the (8, 2, 2) matrix, and the 
20 sums of the two columns of a pair, constitute the 60 
different base and derived characteristics of the (8, 2, 2) 
matrix in which the first four binary symbols do not all 
vanish. The three additional characteristics of the (3, 2, 2) 
matrix in which the first four binary symbols vanish are 
the two columns of the added twenty-first pair, and their 
sum. Thus, we obtain in all the p” — 1 = 27° — 1 = 63 
characteristics of the (8, 2, 2) matrix. 

It is seen readily that the iteration process just described 
is valid for the passage from any (n, 2, 2) matrix thus 
formed to another lossless (n + 1, 2, 2) symbol coding 
matrix for one-error correction. It is also seen that the 
iteration matrices, which play a part analogous to that 
played by the single symbols of the last line when passing 
from an (n, p, 1) matrix to an (n + 1, p, 1) matrix, con- 
stitute the key to the iteration process. These iteration 
matrices, and some associated concepts, are now examined 
in some detail. 


matrices are all different from each other, ( 


matrices are all different from each other ( 


four biquadratic elements 


IrerATION MATRICES 
1 
Designate by b,, b, and b,b. the two base elements, 0 


and ° 


: : 1 : 
v of the iteration matrices, and their sum, L With 


this convention, the three nonvanishing iteration 


matrices (7) may be written: 
biG; Oi Divs 
Consider now the matrix: 
by nb; 
b, i 


It will be observed that the second and third iteration 
matrices utilized above are the two lines of matrix (11), 
and that the first iteration matrix is the sum modulo 2 
of these two lines. 


and bs bbs. 


(11) 
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TABLE I 


IRE TRANSACTIONS ON INFORMATION 


September 
' 


THEORY 


Master IreratioN MATRICES , 
q=2 q=3 q = 4 q=5 q=6 
12 123 123 1234 1234 EF By ab bs 12345 123456 12345 698 
MAT. 12 12 13 12 14 12 15 12 16 
23 123 23 124 23 125 23 126 9 
IND. 34 1234 34 1235 34 1236 
45 12345 45 12346 
56 123456 
a’s: Q’s: 
Oi i © i @ ik 
@) al @ i i () ib it @ a 
p= 2 MIM MIM MIM MIM MIM B's: B's: MIM MIM 
if itil @ @ iL il it @ @ 
10010 TFORORINO 
i @ a @ i i @ it @ i 
a’s 
12s S22 
(BEER 
i OW O 
10200 
OFOR IO 
a’s: 11070) 092 
i also 
p=3 MIM MIM Bis: MIM MIM MIM a’s: 
li @ (2 2 al 
102 Li @ 2 @ 
B's: 
ORO MIO 
Q) il @ Oil 
LO Oth 
a’s: a’s 
13 142 
p=s5 B's: B's: MIM MIM MIM 
iL 33 120 
OL 
a’s: 
145 
p=iT7 MIM B's: MIM 
105 
133 


Designate by p;; the element at the intersection of the 
ith line and jth column of matrix (11), and let a; and 
8; designate factors which for p = 2 may be 1 or 0. It 
will be verified by inspection that the set of three obser- 
vations made earlier is equivalent to the observation 
that no two sums of the form 

~ Qos 
are identical unless all the a’s and @’s of the two sums 
are also identical. This last observation is also equivalent 
to the observation that no sum of the form (12) vanish, 
unless all es and 6’s vanish. It will be convenient to 
state that matrix (11) fulfils for p = q = 2 the general 
condition: 


(12) 


> a:8;p;; 40 (mod. p) 


when 


Saez Sepa (13) 


OP ea Nee A Os a7 <p, Oe ep 


and any matrix satisfying this condition will be termed 


a “master iteration matrix.’ A set of p’ iteration matrices 
is formed from a master iteration matrix by giving the 
as in >Sa;p;; all p* combinations of values. When 
passing from the (n, p, q) coding matrix to the 
(n + 1, p, g) coding matrix, the (n, p, q) matrix is written — 
horizontally p* times, and each iteration matrix is written — 
(p*” — 1)/(p* — 1) times below each (n, p, g) matrix. The - 
(n + 1, p, g) matrix is completed with a q-tuple column 
consisting of m all-zero matrices above a (1, p, g) matrix 
(7.e.,& q¢ X gq matrix of zeros except for a main diagonal 
of 1’s). 

Table I has been prepared to list, in compact form, 
several matrices which, depending upon the value of p, 
may or may not constitute master iteration matrices 
(MIM). 

All matrices listed in Table I are symmetric with 
respect to their main diagonal, and each sucessive line of 
these matrices is shifted to the left one element with re- 


—_ 


>A master iteration matrix is three-dimensional, since it is a 
two-dimensional array of one-dimensional sequences. Two-dimen- — 
sional matrices may be derived from it with the operation Sta:pi;_ 
and one-dimensional sequences may be derived from it with the 
operation >0*7 a;8;p;;. 


958 


spect to the line above it. They are therefore fully charac- 
serized by their first line and last column. Furthermore, 
he indices only of the g-dimensioned base elements 
, forming the p,;’s of the first line and last column have 
been indicated in the matrix indices (MAT. IND.) lines. 
“or instance, the first of the two matrices at the p = 3, 
y = 3 intersection is the ternary master iteration matrix: 


ty lips ts 
Ped a Ee (14) 
iy til Ute 
ivhere 
i 0 0 1 0 
ty a 0, is — Ike ile = 0, t,t, = iL tats = ie 
0 0 il 0 1 


Giving the a,’s in )la:p;; the successive 27 combi- 


hations of values: 


EO) Cae Og0 
AUD SES” 20 
eee) De tee O 


(oe the 27 successive iteration matrices: 


2295 212, 202 020 010 000 
OT e200." 222, 022, O11, 000. 
291 120 022 202 101 000 


_ When a matrix is not a master iteration matrix, the 
single set or the several sets of values of the a,’s and 
3,’s for which (13) is not fulfilled are listed in ascending 
index order. For instance, the second matrix at the 
p = 3, q = 3 intersection is not a master iteration matrix 
because (13) is not satisfied when 


= 1, “ar = 2, a; = 2 
and when 

| 6. =1, Br =1, 6 =0. 
Alternately, the same a values and 
By = 1, B2=0;, Bs = 2 


Ho not satisfy (13), nor again the same a values (or the 
bv values multiplied by 2 modulo 3) and any linear combi- 
hation modulo 3 of the 6 values given above, such as 


6B, = 2, 6B, = 1, Bz; = 2 


Bi 107) 8s al es — 1, etc. 


- The remarks made above apply to all sets of values of 
she a’s and #’s given in Table I which do not satisfy 
13), and since the matrices of Table I are symmetric 
vith respect to their main diagonal, the a’s and #’s may 
e interchanged in all sets of values given. 
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The case p = 2, q = 5 is particularly noteworthy, 
since it is the only case examined for which a master 
iteration matrix was not found. Six other symmetric 
matrices have been examined, and none has yielded a 
master iteration matrix for this case. The indices of the 
first and last line of the matrices thus examined are: 


Pep sy 5) 


1 : ie Weeks re We aie Sh 
5019 124935. 14 


) © 5, 
5 12 34 15 238 5 123 234 345 145 


{2.8 4 Set 9) a eee 
5 12 123 1234 12345 5 12345 2345 345 45° 5 125 123 234 345. 


It has been noted” that no case of lossless coding is 
known, for which symbol coding is impossible, but the 
failure to find a master iterating matrix for p = 2, q = 5, 
among the eight matrices examined, suggests the interest- 
ing possibility that symbol coding may be proven im- 
possible for this case, which is known to have a close- 
packed code (see Note). 

Consider now the extension of the foregoing coding 
matrix construction to the cases in which the symbol 
order is the square (¢ = 2) of any odd prime. We may, 
without loss of generality, postulate the following matrix: 


be oP (15) 
D2 —bp; 


1 : 
where p, = 0 P2 = . In order to determine under what 


conditions matrix (15) is a master iteration matrix, con- 
dition (13) is applied to (15), and we obtain the require- 
ments 


a8, — 2825 x 0) 
or a Ba ale O28; = 0 


or both. 

If we consider the §’s as the variables with respect to 
which the set of homogeneous congruences of (16) must 
be unsolvable, condition (16) can be replaced by the 
equivalent necessary and sufficient condition that the 
determinant of the coefficients of the $’s do not vanish: 


(mod. :p) (16) 


ay re bas 


= dai + bas #0 (mod.p). (17) 


As aa, 


It is obvious that condition (17) is satisfied whenever 


Gnas, pmo neenl)s 
and 
a, #0, a, = 0, (mod. p) 
or 
a, =0, a #0. 
and we examine now the cases in which 
a#0, 6#0, Godse) 
a, #0, a, #0. 


LO8 IRE TRANSACTIONS ON 
(mod. 4) p cannot divide 

Therefore, if we make 
ax, We obtain 


It is known’ that when p = 
numbers of the form x + 
b = 1 in (17) and set a, 


3 
ie 
sh —— — 
D = a +a; = ax? + 1) 4 0. (mod. p = 4n + +3) 


When we make a = 1, b = 2 in (17), we obtain a 
quadratic expression of the form aj + 2 a3, and since 
numbers of the form x + 2 are not divisible by 
primes’ = 5 (mod. 8), we obtain, setting a; = a2.%: 


D = a + 203 = a(x” + 2) 40. (mod. p = 8n 4+ 5) 


Finally, when we make a = 2, 6b = 3 in (17), we obtain 
an expression of the form 2a} + 3a}, and since numbers 
of the form 2x” + 3 cannot be divided by primes’ of the 
form 24n + 1 or 24n + 17, we obtain, setting a, = apw: 
D = 2ai + 8a; = a3,(2”° + 3) £0. 


(mod. p = 24n + 1, 24n + 17). 


It is seen readily that the last two cases cover all 
primes of the form 4n + 1, and since the case a = b = 1, 
covers all primes of the form 4n + 3, all odd primes are 
covered. Therefore, master iteration matrices can be 
constructed for all squares of odd primes. 

As an example, consider the case p = 5. Condition 
(17) is satisfied when a 1, b = 2, and the matrix 


Vo 


: (18) 
—2y, 


Vy 
V2 


is a master iteration matrix modulo 5. On the other 
hand, matrix (11) is not a master iteration matrix modulo 
5, because (12) vanishes modulo 5 when a, = 6, = 1, 
a, = 3, 8, = 2, and when the p,,’s are the b’s of (11). 

The question of whether the search for master iteration 
matrices for values of q higher than 2 can be systema- 
tized is proposed as an interesting unsolved problem. of 
coding or group theory. 


MisceLLANrEous NovTeEs 


An interesting condition has been utilized by Zaremba’® 
for proving the nonexistence of certain codes. This 
condition will be described in connection with its appli- 
cation to the once surmised close-packed code for correct- 
ing up to two errors in 90-binary symbol messages. These 
numbers satisfy the condition previously published’ that 
the first three numbers of the corresponding line of 
Pascal’s triangle add up to a power of 2: 


90.89 


12 
Lire Onan) oe 2s 


If such a code were to exist, the code book would contain 


2’* code messages with the minimum distance 5, and with 
the allowable assumption that the all-zero message 


6 T. Nagell, ‘Introduction to Number Theory,’ John Wiley and 
Sons, Ine., New York, N. Y., p. 185; 1951. 

Uae iy NBO, 

8 Tbid., p. 190. 

* Private communication. 
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belongs to the code book, the remaining code messages 
with the greatest number of 0’s would contain 85 0’s and 
5 1’s. For the code to be close packed, every message with 
87 0’s and 3 1’s should be obtainable in only one way by 
replacing 2 0’s by 1’s in the 85-0 and 5-1 code messages, 
and this condition determines the number of the 85-0 
and 5-1 code messages. A single error in the 0’s of these 
would produce 86-0 and 4-1 messages, and the number 
of remaining 86-0 and 4-1 messages determines the 
number of 84-0 and 6-1 code messages. This procedure 
can be extended to determine all the numbers of code 
messages with an increasing number of 1’s. The numbers 
thus found should be integers, and this necessary but not 
sufficient condition for the existence of a close packed 
code constitutes Zaremba’s condition. In the particular 
instance examined here, it is found that the number of 
83-0 and 7-1 code messages is fractional, and this demon- 
strates anew the impossibility of the close-packed code 
surmised above. 

When Zaremba’s condition is applied to the singular 
3-error correcting code with 12 binary information | 
symbols and 11 check symbols built with the matrix 
previously published, it is found that there are: 1, 253, 
506, 1288, 1288, 506, 253 and 1 messages with 0, 7, 8, 11, | 
12, 15, 16 and 23 1’s, respectively. 

Of course, the addition of a check digit serves to cluster | 
the code messages even more into 1, 759, 2576, 759 and 1 
message with 0, 8, 12, 16 and 24 1’s, respectively. | 

The application of Zaremba’s condition to the other — 
singular 2-error correcting close packed code with 6 } 
information and 5 check ternary symbols shows that ; 
there are 1, 132, 132, 330, 110 and 24 code messages with | 
11, 6, 5, 3, 2 and no 0’s, respectively. 

The application of Zaremba’s condition to the hypo- 
thetical 1 error correcting code with 5 information and | 
and 2 check symbols of order 6 does not disprove the 
existence of this code, for we find that the code book — 
should contain 1, 175, 525, 1890, 3010 and 2175 code 
messages having, respectively, 7, 4, 3, 2, 1 and no 0’s. 

It is not known at present whether this code exists or 
not, but if it exists, it can be shown not to have the | 
following property exhibited by the (2, 2, 2) code tabulated 
earlier. 

Consider any code message, and leaving unchanged 
any 0’s it may have, add respectively 1 and 2 modulo 3 | 
to all other symbols, in the number system of order 3 
with the symbols 1, 2 and 3. The two new messages thus 
obtained form part of the code (for instance, from the | 
code message 01323 we derive the other code message 
02131 and 03212). H| 

Consider now the hypothetical (2, 6, 1) code, and 
select the greatest number of code messages in which 
three given symbols are 0’s. The most favorable assump- 
tion, for ease of code message “packing,” is that this 
number be the same for all similar groups, and be equally 
broken down into code messages in which the other four 
positions are filled by three or four nonzero symbols. 
We obtain thus 525/35 = 15 messages with no 0’s for 
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Instance, in the first four positions, and 175/35 = 5 L-=0 0 

messages each with. a single 0 in any one of these first an tae 4 1 0 in system 
four positions, with 0’s in the last three positions for all. 9” 7” = 7% BBE oer EES pt aah ot orderine 
_ Postulate now that to each message corresponds 4 0 O 1 
additional messages obtained by adding 1, 2, 3 and 4 ty, tx, --- , tg = same base elements in system of order 3". 
modulo 5 to the nonzero symbols, in the number system 1, v2, ++ , Ve = same base elements in system of order 5’. 
of order 5 in which the symbols are 1, 2, 3,4 and 5. We p,, Po, -* + , pp = Same-base elements in system of order p’. 
may assume, without loss of generality, that the code a; and £; = one-dimensional factors in system of 


messages selected by this process are: 


Maid sl eden 0:40 
Mette syeds 01,0; <0 
| ay ume xen! a 020 
| erin: 0250 8 O)ss0) 
ie 51073 0 -0-<0 
ik Se Se 0 Oe) 
Opec <0" 20.40 


and the 28 other messages obtained by adding 1, 2, 3 
and 4 modulo 5 to the nonzero symbols. If the code exists, 
it should be possible to replace the 2’s by numbers which, 
‘for the whole rectangle, satisfy the condition that no 
four nonzero numbers exist at the apex of any rectangle, 
‘which have the property that the difference (mod. 5) of 


‘any two on a side is equal to the difference of the other 
‘two. This condition is imposed by the requirement that 
‘the minimum distance 3 exists between any two for the 
135 code messages involved. It has been verified by in- 
‘spection that this condition cannot be realized. Hence, 
if the (2, 6, 1) code exists, it cannot have the property 


postulated above. 


List oF SYMBOLS 


p = prime number base. 

q = power of p in coding system of order p’. 
n = number of redundant digits in system 
| of order p’. 

((n, Dp, @) = characterization of lossless code or 


coding matrix in system base p’ with 
n redundant digits. 


WX on = redundant check digits in system of 
i order p. 

IY, = information digits in system of order p. 
JE, = check digits calculated after reception 
in system of order p. 

ee = redundant check digits in system of 
| order p*. 


= information digits in system of order p*. 

= check digits calculated after reception 
in system of order p’*. 

= q-dimensional term in matrices (whether 
master iteration matrices or not). 


order p. 
DEFINITION OF TERMS 


Close-packed code: An. e-error correcting code in which 
the number of words is given exactly by the expression: 


qn 


q 


> (p* — 1)’ 


ee 
D5 4 
; 

Sum (p,p. of ensembles p, and p.): The ensemble formed 
by the addition modulo p of the homologous symbols 
of the individual ensembles. The symbol @, sometimes 
used to designate such sums, is not used here in order 
to conserve space, as no confusion results from this 
convention. 

Coding matrix: The matrix formed by the coefficients of 
(4) or (5). 

Iteration matrix: One of p* g-dimensional square matrices 
which are required to pass from the (n, p, qg) coding 
matrix to the (n + 1, p, qg) coding matrix. 

Master iterating matrix: A q-dimensional square matrix 
composed of g-dimensional elements, from which the 
p’ iteration matrices can be derived systematically, 
by the operation >)'=% a:p;;. 

Corrector: The ensemble of nq digits of order p obtained 
as a result of the checking operations defined by (4) 
and (5). 

Base characteristic: The ensemble of nq digits of order p 
forming any column of a coding matrix below an X,,; 
or a Y,;, (also simply the “characteristic”? when p = 2, 
Gane 

Derived characteristic: Any sum of the base characteris- 
tics below X,,; 8 or Y,,s.@ or 7 = 1) 2)0--= gg), with 
one of the weights 0, 1, --- , p — 1 given to each base 
characteristic. 

Note: After the submission of this article, Prof. F. L. 
Dennis discovered and communicated to the writer an 
MIM for the case p = 2, g = 5. Like the matrices listed 
in Table I, Dennis’ matrix is symmetric with respect to 
the main diagonal, and its successive columns are shifted 
upward one element with respect to the preceding 
columns. Thus, Dennis’ matrix is fully characterized by 
its first and fifth lines, the indices of which are: 


12a 23 ed 
5 13 24 35 134 


——= 
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On Sampling the Zeros of Bandwidth Limited Signals: 


¥F, E, BOND? anv C. R. CAHN 


Summary—tThe sampling theorem enables a band-limited signal 
to be expressed in terms of a set of sample point values, which 
occur at the Nyquist rate. The sampling theorem has been general- 
ized to include nonuniform sampling and the use of derivatives of 
the signal. In the present paper, a sampling theorem has been 
developed which utilizes information related to the zeros of the 
signal. The concept of complex zeros is introduced to show that the 
zeros occur at the Nyquist rate. This sampling theorem can be of 
use for enabling the transmission of binary signals, such as fac- 
simile and infinitely-clipped speech, over a continuous band-limited 
channel. The result indicates the desirability of developing a com- 
pletely general theory of sampling applicable to the various situ- 
ations which may arise in practice. 


ROLE OF SAMPLING IN COMMUNICATIONS 


) { ODERN communication theory has presented 


the fundamental requirement for matching the 

information source to the communication channel 
to achieve optimum efficiency in transmission. Although 
the ‘‘matching”’ is usually assumed to pertain to the 
statistical constraints of the message, it also may be 
interpreted in a broader sense to include the type of 
signal structure. Since the channel can be either con- 
tinuous or discrete, as is true for the information source, 
the need for conversion from each form to the other often 
arises in many practical situations. Examples of each type 
include: the amplitude sampling of speech at uniform 
intervals prior to transmission over a microwave relay 
system employing pulse position modulation and the 
transmission of on-off or polar telegraph signals over a 
cable system. These types of conversion are accomplished 
by the sampling theorem,’ which states that a function 
of time f(t) which is band-limited in (0, W) is uniquely 
determined by a knowledge of its ordinates at uniform 
intervals 1/2W seconds apart. Furthermore, for any 
arbitrary set of such sample points, a continuous function 
exists, band-limited in (0, W) and passing through all 
points. The theorem may be expressed as 


= n \sin r(2Wt — n) 
i ge (27) ON aan ie (1) 


Applications of the theorem often involve sampling 
over a finite interval of time 7’. Consideration of only a 
finite number of samples causes an interpolation error 
which can be quite large at the edge of the interval. This 
error inside the interval can be reduced as the 2WT 
product is made large. 

Other types of communication systems must deal with 
discrete signals (either at the source or in the transmission 


* Manuscript received by the PGIT, February 14, 1958. 

+ The Ramo-Wooldridge Corp., Los Angeles, Calif. 

10, E. Shannon, ‘A mathematical theory of communication, 
part III,’ Bell Sys. Tech. J., vol. 27, p. 627; October, 1948. 


channel) occurring at irregular intervals. An example is 
the application of nonsynchronous time division multi- 
plexing of speech channels.” This implies amplitude — 
sampling at nonuniform but arbitrarily specified intervals, 
with theoretical justification provided*’ by the use of — 
the Lagrange interpolation functions expressed as | 


N 


i) = Do HEAD Q) 
where 
g(t) 
EMD) = aap 
and 


a) = TI (1-2). 


n=— 20 


The instants of sampling ¢, are arbitrary except that 
the average spacing between successive instants is 1/2 W. | 

Yen’ has shown that the interpolation functions, L,(é), | 
are band-limited so long as the number of nonuniform | 
intervals are finite. Paley and Wiener’ prove the validity | 
of the expansion with an infinite number of nonuniform | 
sampling intervals but with strong restrictions in the / 
location of the sampling points. Thus, it is seen that the ) 
expansion expressed by (1) is actually a special case of the 
Lagrange interpolation formula. 

Still other applications require the measurement of ' 
two or more quantities (for example, amplitude and 
slope) at sampling points separated by two or more 
Nyquist intervals. For this application, Jagerman and 
Fogel® have shown that 


00 = © LG) (a) ee 
(3) 


It is to be noted in all cases that the average rate of 
sampling cannot be less than 2W per second (the Nyquist | 
rate). 


| 


_?J. R. Pierce and A. L. Hopper, ‘““Non-synchronous time divi- | 
sion with holding and random sampling,’ Proc. IRE, vol. 40, pp. | 
1079-1088; September, 1942. 
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vol. IT-2, pp. 139-146; December, 1956. | 


Tur Nerep For Imp.ricir SAMPLING 


All of the sampling procedures described above are 
issociated with a set of sampling points ¢, which are 
independent of f(t). Some communication problems 
require the sampling points ¢, to be determined by the 
characteristics of f(). An example of such a case is 
rompted by the discovery’ of the relatively small effect 
of infinite clipping on speech intelligibility. This suggests 
the possibility of transmitting a continuous speech signal 
F(t) over a discrete channel by preserving the occurrence 
of the zero crossings and nothing else. The reverse situa- 
ion occurs when it is desired to transmit black and white 
aa over a continuous channel. Here, the information 
source is a discrete (binary) signal with abrupt zero 
erossings at random points in time. Maximum conservation 
f bandwidth may be achieved by generating a band- 
imited signal which crosses the zero axis in step with the 
inary signal. After transmission over the continuous 
thannel, the source signal could easily be reconstituted 

y infinite clipping, although to obtain this minimum 

ossible transmission bandwidth, the continuous channel 
may be required to handle much greater peak power than 
er 


ess 
| 
] 


GENERALIZED DEFINITION OF ZERO CROSSINGS 


_ The need is thus evident for a sampling theorem which 
leals with the location of the zero crossings, rather than 
the amplitudes (or slopes) at specified instants. A sig- 
hificant characteristic to be considered is the average 
tate of the zero crossings. In general, it is less than the 
Nyquist rate. For example, consider the case of a random- 
noise signal n(t) with a flat spectral density band-limited 
in (0, W). Rice® has shown that such a noise signal will 
bross the zero axis at an average rate of 2Wv/t per 
second. He has also shown that for the mth time derivative 
pf n(t), the average rate of zero crossings will be 

2W ae per second, 


thus indicating that differentiating an infinite number 
bf times would be required to obtain a signal with zeros 
occurring at the Nyquist rate. The zeros of a high-order 
Hee ao 1 of n(t), however, are very closely correlated, 


and no longer represent independent samples. 

| The deficiency in zero crossings is explained in a set of 
theorems by Titchmarsh,’ the interpretation of which 
tan be summarized as follows. Let f(t) be band-limited 
in (0, W), and let V(f) be its (double-ended) Fourier 
transform.?° Now consider the complex function 


7J. C. R. Licklider and I. Pollack, “Effects of differentiation, 
ntegration, and peak clipping upon the intelligibility of speach,”’ 
. Acoust. Soc. Amer., vol. 20, pp. 42-51; January, 1958. 

88. O. Rice, ‘Mathematical analysis of random noise,” Bell Sys. 
ech. J., vol. 24, p. 54; January, 1945. : 

9h. C. Titchmarsh, ‘““The zeros of certain integral functions,’ 
roc. Lond. Math. Soc., vol. 25, pp. 283-802; May 14, 1926. 

10 Strictly speaking an infinite sample of a signal like random 
hoise does not possess a Fourier transform. However, the results 
stablished in this paper can be applied to such functions by a 
uitable limiting process. 
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REAL FUNCTION COMPLEX REPRESENTATION 


Fig. 1—Real and complex representation of Vo + V sin 2r Wt. 


fe) = [vine af 


where zg = ¢ + ju is a complex variable whose real axis 
coincides with the real time axis. [Note: f(z) along the 
real axis coincides with f(¢).] Then f(z) is a real, entire 
function, described by the location of its zeros, which 
are either real or occur in complex conjugate pairs. In 
general, the complex zeros tend to cluster near the real 
axis. Furthermore, the aggregate of zeros (including real 
and complex ones) occur, in the limit, at the Nyquist 
rate. The function thus is expressible by the infinite 
product 


j@) = 10 I1(1 - 2) (a 
where {(0) # 0, z, = R,e’”” is the location of the nth 
Zero, 

Te; SS Rv 
and 
lim 2MiRs I. 
n 


no 


It is hence apparent that continuous. band-limited 
functions generated by natural information sources will 
include ‘“‘complex’’ zeros which, unlike real zeros, are not 
physically detectable. A simple example of complex zeros 
can be given with a signal structure which has no infor- 
mation content but is useful in illustrating the nature of 
the zeros. Consider a single frequency sine wave combined 
with a de bias voltage 


f(t) = Vo + V sin 2n Wt. 


For V> = 0 the real zeros of f(t) occur at regular intervals 
1/2W seconds apart. As V> increases, the zeros of f(t) will 
start to converge in pairs, ultimately forming second- 
order zeros when |V.| = V. (See Fig. 1.) If |Vo| > V, then 
f(t) will never vanish. However, if ¢ is generalized to a 
complex argument z = ¢ + ju, then it will be found that 
f(z) admits a uniform distribution of complex zeros 
located at 


Nese neet aa 
(n+ 5) sty j cosh | V 
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where n includes all odd or all even integers depending 
on whether V,/V is positive or negative. Note that the 
larger de biasing signal will move the complex zeros 
further from the real time axis. For a more general signal 
band-limited in (0, W), the distribution of complex 
zeros will be irregular (except for symmetry about the 
real time axis) and as yet no procedure has been formu- 
lated for locating them from the knowledge of the con- 
tinuous function. 


SYNTHESIS OF A BAND-Limtirep FUNCTION 
WITH A GIVEN SET OF ZEROS 


The preceding discussion has indicated that a band- 
limited function can be expressed by (4) in terms of 
its real and complex zeros which occur in the limit at the 
Nyquist rate. The formula, however, requires the knowl- 
edge of all past and future zeros, both real and complex, 
to reproduce f(t). A more practical peas is that of 
representing a band-limited function over a finite interval 
in terms of the zeros which occur within this interval. If 
the contributions of the zeros outside of the specified 
interval are ignored, the resulting finite continued product 
will be a high-order polynomial and will have gross 
distortion near the edges of interval. An exponential 
correction of the amplitude has been discussed'’ based 
on the assumption that the zeros outside of the interval 
of interest are real and occur on the average at the Nyquist 
rate. This assumption is justified by the theoretical result 
that the zeros tend to cluster along the real axis for any 
band-limited function. 

Another representation, which can be more easily 
related to the sampling theorem (1), is obtained by making 
the reasonable approximation that outside of the specified 
interval the zeros are all real and occur exactly at the 
Nyquist rate, while inside the interval they occur at 
slightly less than the Nyquist rate. As with (4), the 
function, for simplicity, is required to be nonzero at 

= (0. Let the interval in question extend from — 7/2 to 
T/2, and let N be the largest integer not exceeding 7W. 
Outside of the interval, the zeros are assumed to occur 
at times equal to + n/2W, forn = N +1,N42,.-- o, 
and inside there are a maximum of 2N real or complex 
ZeLOS Z = t, + ju,, satisfying |t,| — 7/2. From (4), 
the band-limited function may be expressed as 


{ I] — if) ath fr (ey 
sar I] (1 — t/z,) 
=f gh 8 r E = Bus, | 


n=1 
in which is used the well-known infinite-product repre- 
sentation of the sine. The product in the numerator is 
extended over values of m corresponding to the zeros 2,, 
within the interval. 


() = 


sin 21Wt (5) 


uF. E. Bond, “The information-bearing elements in bandwidth 
limited signals,’’ Doctoral dissertation submitted to Polytechnic 
Institute of Brooklyn; June, 1956. 
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With the limitation on the number of zeros of f(t) 
within the interval in question, the factor multiplying. 
sin 2x Wt in (5) is a proper rational fraction and may be 
expressed as the partial fraction expansion 


IT — t,) a 
10) oe ee + aa 
Q2rWt al (1 + =— 2Wt wi\( —— san ea WE WN | 
n=1 nN (6): 
where 
ll owe I] Wz, — n) 
FF m m 

pe tse = 2 ; (7) 

ito TL m= 

mMAn 
From (5) and (6), the result is obtained that 
1) = S Ansin 29W 


neon 2rWt — xn 


N 


>» (-)"4, 


n=—N 


sin (24Wt — xn) 
2nWt — an 


which is seen to have exactly the form of (1) with a finite 
number of terms. Thus, for a specified set of real and 
complex zeros within an interval, a band-limited function 
can be derived having those zeros within the interval and 
having real zeros exactly at the Nyquist rate outside the 
interval. 

The amplitudes at the sampling points, given by (Ty 
may be expressed in terms of the migrations of the zeros 
from the sample point locations, thereby giving a clearer 
interpretation of the result. Let the migration of the nth 
zero be given by 


A, = 4, — n/2W 


(= z, — n/2W for complex zeros). 


Note that for at least one value of n, no zero is associated 
with the sample point and no zero migration is defined. 
Disregarding the fixed scale factor, the amplitude of the 
nth sample point is i 


(1) Al ie Dears N 
iba (m — n) 
m=—N 
man (10) 
2WA 2WA 
ae (— il n n (1 m 
) I] (m-7n) I] zi m—n 
m for which 
for which zeros are 
no zeros are associated 
associated. men 
mAn 


where the external 2WA,, is replaced by unity if no zero’ 
is associated with the sample point. This result shows 
that the location of a zero (specifically the migration from! 
a sample point location) affects the amplitude of f(d) in’ 
its immediate vicinity but does not have a marked effect 
on the signal at a much earlier or later time. A large’ 
migration, resulting in a large interval between successive 
zeros, will produce a large signal amplitude. 


958 


_ As mentioned earlier, the complex zeros, in general, are 
10t specified or known. For example, if a binary signal, 
uch as infinitely-clipped speech, is to be replaced. by a 
and-limited signal having the same zero crossings, only 
he real zeros are given. To assume that complex zeros 
xist in addition to the specified real zeros would increase 
he bandwidth of the resulting band-limited signal. 
Nevertheless the complex zeros may be introduced to 
educe the large signal amplitude which otherwise would 
esult when there is a large spacing between successive 
eal zeros. This is an example illustrating qualitatively 
\s bandwidth may be exchanged for signal-to-noise 
atio when information is to be transmitted. 

A simple illustration of the theory will now be given. 
uppose that the zero crossings of the binary signal within 
lhe interval 7 in Fig. 2 are to be transmitted. The dotted 
‘xtension of the given signal shows how real zeros are 
“ssumed to occur at a uniform rate outside of the specified 
interval. Since there are four zeros in the interval, a band- 
width W is chosen so that TW is not less than 4/2 = 2. 
(he zero-crossing migrations from the sample points are 


2WA_, = 0 
2WA, = +} 

SW Medes 1 
| 2WA, =0 


ind no zero is associated with the sample point at —1. 
he calculated band-limited function having the specified 
Jero crossings is shown in Fig. 3. As expected, the relatively 
bng spacing between the zeros at —2 and 0.25 contributes 
‘0 a large amplitude in this region, and a channel with a 
ligh peak power capacity is required to transmit the 
ignal. 

CONVERSION OF “MIGRATION” OF ZEROS TO 

AMPLITUDE INFORMATION 


_ Although the A,’s have been shown to provide the 
inique definition of a band-limited signal (vanishing at 
1/ 2W + A,) by means of a straightforward formula, the 
omplexity involved would tend to inhibit practical 
pplication. However, for the problem of transmitting 
he abrupt crossings of a discrete binary signal over a 
jontinuous channel, it would be relatively simple to 
‘onvert the migration intervals A, to pulse amplitudes 
iccurring at uniform intervals (1/2W) with the individual 
‘mplitudes proportional to A,. In this case, 2W is the 
iverage number of crossings per second and the minimum 
heoretical transmission bandwidth remains W. The 
eo could consist basically of linear-sweep 
‘enerators which measure the intervals of time between 
ie occurrences of the zero crossings and the sampling 
istants, as determined by the average rate of crossings 
kee Fig. 4). Conversion of the pulse amplitude sequence 
lack to the original binary sequence with crossings at 
/2W + A, would be possible at the receiving end of the 
jhannel with similar instrumentation. Again, the dynamic 
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Fig. 2—Binary signal. 
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Fig. 3—Band-limited signal having the specified zero crossings. 
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Fig. 4—Conversion of the migration of zero crossings of discrete 
signal to pulse amplitudes at uniform intervals 


PAM Signal 


range of the continuous signal would depend upon the 
distribution of intervals between crossings. If the nature 
of the discrete signal source were such that long intervals 
could occur between crossings, special precautions would 
be necessary to prevent the continuous signal from growing 
to a prohibitive large amplitude. 


NEED FoR A GENERALIZED THEORY OF SAMPLING 


It has been established that a signal band-limited in 
(0, W) and essentially zero outside of an interval of 
duration 7’ can be reproduced from a knowledge of 2WT 
samples (except in the vicinity of the ends of the interval 
where low accuracy may exist). The samples can include 
amplitudes at specified instants or, as is shown in this 
paper, the location (in the time domain) of the generalized 
zeros of the signal. Other theoretical methods of repro- 
duction include the use of the 2W7T Fourier Series com- 
ponents and the first 2W7 derivatives at a single point. 
The types of sampling discussed here appear as logical 
requirements in certain practical communications prob- 
lems, and are by no means intended to be exhaustive. 

The various types of sampling may be viewed as aspects 
of a generalized theory of sampling, which is capable of 
covering all situations that may be encountered. Moreover, 
a generalized theory should be able to handle the practical 
case where the signals are not truly band-limited, since a 
sharp frequency cutoff is realizable only approximately 
with practical networks. Such a generalized theory has 
not yet been found, and a major breakthrough, compar- 
able with the development of information theory itself, 
may be necessary. 
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Enhancement of Pulse Train Signals by Comb Filters’ 


JANIS GALEJSt 


Summary—tThe relative performance of different types of comb 
filters is investigated in conjunction with signal and noise types 
similar to those expected in radar applications. The filter types con- 
sidered are idealized filters with zero transmission stop bands 
between their pass bands, optimum filters maximizing the peak 
signal-to-rms-noise ratio, cascaded delay line filters, feedback type 
filters, and storage tube filters. The pulse train signals consist of 
rectangular or sin x/x pulses with rectangular or sin x/x pulse 
envelope shapes. Power spectra of noise considered are rectangular 
and triangular. With a given number of signal pulses, the perform- 
ances of the different filters vary from each other only by a few 
decibels in most cases analyzed. Storage tube filters exhibit lower 
signal-to-noise power ratios, but higher peak signal-to-rms-noise 
ratios, than the feedback type filters. Inaccurate delay times of 
filter delay lines are shown to decrease the peak signal output more 
than the signal power output and to affect the cascaded delay line 
filter less than the feedback type filter. Correlation techniques are 
compared with comb filters. The crosscorrelator exhibits the same 
peak signal-to-rms-noise ratio as the optimum filter. 


I. INTRODUCTION 


( pa FILTERS for enhancing the detection of 
pulse train signals in additive noise and optimum 
filters which maximize the signal-to-noise ratio 

at a predetermined time instant have been the subjects 

of numerous investigations,” most of which consider 
one or a few filter types. The pulse-to-pulse integration, 
which characterizes the operation of comb filters, also 
takes place in storage tubes.'°’* Correlation tech- 


* Manuscript received by the PGIT, April 29, 1958. This paper 
is based on portions of a dissertation submitted in partial fulfillment 
of the requirements for the Ph.D. degree in electrical engineering, 
Illinois Inst. Tech., Chicago, Ill., January, 1957. 

Appl. Res. Lab., Sylvania Electric Products, Inc., Waltham, 
Mass; formerly at Cook Research Labs., Chicago, Ill. 

1P), O. North, ‘‘An Analysis of the Factors which Determine 
Signal-Noise Discrimination in Pulsed Carrier Systems,’’ RCA 
Labs., Rep. PTR-6-C; June, 1943. 

2B. M. Dwork, ‘‘Detection of a pulse superimposed on fluctua- 
tion noise,’”’ Proc. IRE, vol. 38, pp. 771-774; July, 1950. 

31, A. Zadeh and J. R. Ragazzini, ‘Optimum filters for the 
detection of signals in noise,’ Proc. IRE, vol. 40, pp. 1223-1231; 
October, 1952. 

4S. F. George and A. S. Zamanakos, “‘Comb filters for pulsed 
radar use,’ Proc. IRE, vol. 42, pp. 1159-1165; July, 1954. 

5 A. E. Bailey, “Integration in pulse radar systems,” in “‘Com- 
munication Theory,’ papers of 1952 London Symposium, W. 
Jackson, ed., Academic Press, Inc., New York, N. Y., pp. 216-230; 
1953. 

6 J. Freedman and J. Margolin, ‘‘Signal-to-Noise Improvement 
through Integration in a Delay-Line Filter System,’’ Lincoln Lab., 
M.I.T., Lexington, Mass., Tech. Rep. No. 22; May 13, 1953. 

7G. I. Cohn and G. T. Flesher, ‘Investigation Pertaining to 
Elimination of Ambiguities Due to High Pulse Repetition Rates,” 
Sixth Quart. Prog. Rep., Signal Corps Contract No. DA-36-039 
SC-56696, pp. 28-45; July 19, 1955. 

8R. E. Graves, “Design and Analysis of an Optimum Comb 
Filter Based upon an Information Storage Device,’’ Goodyear Air- 
craft Corp., Akron, Ohio, Rep. GHRA-20; January, 1953 

°R. E. Graves, “Miscellaneous Considerations in Comb Filter 
Design,’’ Goodyear Aircraft Corp., Akron, Ohio, Rep. GERA-24; 
August, 1954. 

10 J. V. Harrington, “Storage of small signals on a dielectric 
surface,” J. Appl. Phys., vol. 21, pp. 1048-1053; October, 1950. 

ny Harrington and T. F. Rogers, “Signal-to-noise improve- 
ment through integration in a storage tube,’’ Proc. IRE, vol. 38, 
pp. 1197-1203; October, 1950. 


niques’ can be thought of as the counterpart of comb 
filters in time domain,” although their principles of 
operation are basically different. 

An objective of this paper is to classify the relative 
improvements possible in detecting several types of 
pulse train signals in additive noise by various comb 
filters. The performance of theoretically optimum filters 
is compared with that of simpler and more practical | 
filters. From the problems inherent in the design of comb 
filters only the effects of delay line tolerances are con- 
sidered here. Also, the merits of autocorrelators and cross- 
correlators relative to comb filters are determined. 


Il. GENERAL CONSIDERATIONS 


In most analyses, the filter input is assumed to consist 
of rectangular pulses of constant amplitude. With a 
finite bandwidth and with phase distortions in the trans-. 
mission system, the input pulses of the comb filter will, 
be more or less distorted.’” The rounded-off peak and the 
sloping sides of a heavily distorted pulse may be approxi-_ 
mated by a sin «/x shaped pulse. The envelope of a pulse’ 
train received in a radar set depends on the antenna 
characteristics. The envelope can be approximated during 
the main lobe of the antenna pattern by a sin w/a curve 
as shown in Fig. 1. The signal output of the comb filter’ 
during the main lobe, which is of principal interest for 
computing the signal-to-noise improvement figures, is 
dependent only to a small degree on the signal during the 
side lobes. The spectrum of a pulse train with sin «/x 
pulses or pulse envelope has a rectangular spectrum 
envelope or rectangular spectral bands. This leads to 
expressions readily integrable in the subsequent analysis. 
The approximation of the actual pulse shapes or pulse 
envelopes by a sin 2/x curve introduces a similar error in 
all the filter types analyzed, which is not expected to 
obscure the relative merits of the different filters. 

Assuming a rectangular pass band of the IF amplifier, 
the noise of the radar set can be represented by band- 
limited white noise in the IF stages of the receiver. For 
small signal-to-noise ratios and for both linear and 
quadratic second detector characteristics, this noise 
spectrum is transformed into a triangular noise spectrum 


2Y. W. Lee, T. P. Cheatham, and J. B. Wiesner, “Application 
of correlation analysis to the detection of periodic signals in noise,” 
Proc. IRE, vol. 38, pp. 1165-1171; October, 1950. 

18 R. M. Fano, “Signal-to-Noise Ratio in Correlation Detectors,” 
Res. Lab. of Electronics, M.I.T., Cambridge, Mass., Tech. Rep. 
No. 186; February 19, 1951. 

14H. Davis, “Radar problems and information theory,’ 1953 
IRE Convention ReEcorp, pt. 8, pp. 39-45. 

16 J. L. Lawson and G. E. Uhlenbeck, “Threshold Signals,” 
M.I.T. Rad. Lab. Ser., McGraw-Hill Book Co., Inc., New York, 
N. Y., no. 24, p. 209; 1950. 
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Fig. 1—Radiation patterns of parabolic reflector antennas—skin 
tracking. 


after the second detector.’® Although band-limited white 
noise represents the input noise of comb filters operating 
before the second detector, a triangular input noise 


_ spectrum should be considered for comb filters operating 
_ after the second detector. 


Signal-to-noise ratios do not give a quantitative indi- 
cation of the expected false alarm or incorrect dismissal 
probabilities, which serve as absolute criteria of the 


_ performance of a filter in conjunction with a threshold 


device. Nevertheless, signal-to-noise ratios can be used 


to determine the relative performance of different types 
of filters. The signal-to-noise ratios considered as criteria 
of filter performance here are 


1) Signal power to noise power ratio, 
2) Peak signal voltage to rms noise voltage ratio. 


_ These ratios are defined in Appendix I to establish the 
nomenclature for the comb filter analysis. 


III. Finrers Havine Puysicatty NONREALIZABLE 
ComB CHARACTERISTICS 


The physically nonrealizable comb characteristics of 
the filters considered have infinite attenuation outside the 
pass bands. The pass bands of these filters are 


1) Rectangular, 
Dyaesine/ a: 
The pass band envelopes considered are 
1) Rectangular, 
2 Sisal, 
3) Distorted sin x/x in an optimum filter approxi- 
mation. 


16 Ibid., pp. 158-161. 
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Such filters have been investigated by George* for rec- 
tangular pulses and pulse envelopes, and for noise of flat 
spectrum. He neglected the energy overlap between 
adjacent pass bands. 

The pass bands of filters are centered about the Pulse- 
Repetition Frequency (PREF) harmonics. The spread of 
the signal energy about the PRF harmonics is inversely 
proportional to the number of pulses in the pulse train. 
When the width of the pass bands is 2/m7’, where m is 
the number of pulses in the pulse train and T is the pulse 
repetition period, approximately 90 per cent of the signal 
energy about each PREF line is within the pass band of the 
filter. For a small number of pulses in the pulse train, the 
spectra about the PRF lines are relatively wide, and some 
of the energy of the signal about a PRF line will fall 
within the adjacent pass bands of the filter.‘’ As seen 
from Fig. 2, the per cent error in the signal power compu- 
tation resulting from the neglect of the sideband overlap 
is less than one per cent for the number of pulses, m, 
larger than six. 

Values of the signal power to noise power improvement 
figures, M,, of the filter types considered are listed in 
Table I (next page). Neglecting the overlap of the signal 
sidebands in the signal computations is equivalent to set- 
ting f(m) of (9) equal to zero. Only slight changes occur 
in M, by changing the input noise spectrum from triangu- 
lar to rectangular. The M, of the filters investigated for tri- 
angular noise differs by less than 1 db from the M, values 
of the corresponding filters arrived at by George’ in the 
case of rectangular noise. 


IV. Finrrers Havine Puysicatty REALIZABLE 
ComB CHARACTERISTICS 


The physically realizable filters have finite attenuation 
stop bands between the pass bands. These filters may be 
subdivided into 


1) Optimum filters that maximize the instantaneous 
signal-to-rms-noise ratio, 


17 This partial overlap of signal energy is apparent from (20) of 
George and Zamanakos, op. cit. 
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TABLE I 
Sranat Powrr to Norse Power AND Peak Srqnat Vourace TO RMS Vorrace IMPROVEMENT Ficures or Comp FILTERS 
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5 & | M, — 10 log m, 0.7 | =O. 8 —0.95 | —4.36 | —4.26 


Note: For sin x/zx pulse train envelopes, the physically realizable 
point after the peak of the pulse envelope. 


2) Cascaded delay line filters, 

3) Feedback type filters, 

4) Feedback type filters modified to approximate the 
optimum filters, 

5) Storage tube filters. 


Characteristics of optimum filters that maximize the 
peak signal-to-rms-noise improvement figure, M,, at a 
predetermined instant of time, tf), have been arrived at 
by North,’ Dwork,’ and Zadeh and Ragazzini.* However, 
the signal-to-noise improvements of optimum filters 
relative to nonoptimum filters have not been explicitly 
stated. The transfer functions of the optimum filters 
considered in this paper are computed according to the 
method outlined by Zadeh and Ragazzini.’ The resulting 
optimum filters consist of a cascade connection of a 
noise-shaping network, a single pulse filter, a non-feedback 
type comb filter, and an output delay line (see Figs. 3 and 
4). The noise-shaping network has a transfer function 
equal to the reciprocal of the noise power spectrum, Py(w). 
The noise-shaping network tends to emphasize the spectral 


filters are designed to have the peak signal output at the half-power 


components of the signal where the noise spectrum has 
its lowest amplitudes. The single pulse filter changes the 
shape of the rectangular pulses into triangular pulses of 
double width. The single pulse filter is identical with 
North’s optimum filter for a single rectangular pulse in 
white background noise. The comb filter sums the indi- 
vidual pulses of the pulse train where the pulses are 
weighted in proportion to their amplitudes. The delay 
time of the output delay line depends on the time, 6, 
when M, is maximized. This delay line does not affect the 
peak signal, signal power, or noise power outputs. The 
signal-to-noise improvement figures of optimum filters 
are of the form given by (9), (10), (18), and (19), with 


(1) 


The numerical values of the improvement figures are seen 
from Table I. For sin Ct/Ct pulse envelopes, only the 
optimum filters maximizing /, at the peak of the pulse 
envelope or at the half power point following the peak 
of the pulse envelope are considered. In both: cases, pulses 
starting with the half power point prior to the peak of 


f(m) = 10 log (1 + 0.5m"’). 
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Fig. 3—Filter maximizing signal-to-noise ratio at or after the trail- 
ing edge of the last pulse of a uniform pulse train. 
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Fig. 4—Filter maximizing signal-to-noise ratio at the kth pulse after 
the peak of the (sin Ct/Ct) pulse envelope. 


the pulse envelope contribute to the signal output at the 


instant when M, is maximized. If the signal-to-noise 


ratio is maximized at the half power point of the signal 
envelope after the envelope peak instead of at the envelope 
peak, values of M, are raised 4.8 db and values of M, are 
raised 3 db. However, the higher signal-to-noise ratio 
requires twice the number of delay lines in the filter. 
Cascaded delay line filters have been investigated by 
Bailey,’ who compares them with feedback type filters. 
The cascaded delay line filters are equivalent to the 
optimum filters after the single pulse filter and the noise- 


_ shaping networks are removed (see Fig. 5). Since f(m) of 


(9) is of the same form as for the optimum filters in (1), 
the signal-to-noise improvement figures of the cascaded 
delay line filters differ from the signal-to-noise improve- 


ment figures of the optimum filters only by the constants 


listed in Table I. 

The simple feedback type filter consists of a delay line 
in a feedback path of an amplifier, as shown in Fig. 6. 
With the delay time equal to the pulse repetition period, 


the input pulses are added to the pulses circulating in the 
feedback loop. Stability considerations restrict the gain 


| 


of the amplifier, A, to less than unity. Such filters have 


been analyzed in conjunction with rectangular pulses of 


constant amplitude. 


Dh 


The signal-to-noise improvement figures, M, and M,, 


of the feedback type filter are listed in Appendix II. The 


peak signal-to-rms-noise improvement figures M, are 
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Fig. 5—Cascaded delay line comb filter. 
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Fig. 6—A feedback type comb filter. 


plotted in Figs. 7 and 8 for rectangular and (sin Ct/Ct) 
pulse envelopes, respectively. The signal-to-noise improve- 
ment figures, M, and M,, can be changed to the form of 
(9), (10), (18), and (19) after relating the amplifier gain 
to the number of pulses in the pulse train, as in (28) and 
(29). As seen from Table I, the signal-to-noise improve- 
ment of the feedback type filter is within 2.5 db of the 
optimum filters for uniform pulse trains, but within 
approximately 6 db for (sin Ct/Ct)-shaped pulse train 
envelopes. 

The simple feedback type filter may be modified to 
approximate the characteristics of an optimum filter. 
It is cascaded with a single pulse filter and with a noise- 
shaping network. This makes the pass band envelope of 
the filter identical with the pass band envelope of the 
optimum filter. The individual pass bands, however, 
retain the shape of the pass bands of a simple feedback 
type filter. The resulting signal-to-noise improvement is 
seen in Table I. 

There is a similarity between the data storage process 
in a storage tube and in a simple feedback type filter. It 
can be shown’’''' that the storage of the charge dis- 
tribution over one sweep decreases its amplitude by a 
factor e ”. A similar dependence of the amplitude of the 
stored pulses on storage time is observed in feedback 
type comb filters. Therefore, the integrating process of 
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7—Peak signal-to-rms-noise improvement figure of a simple 
feedback type comb filter. 
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Fig. 8—Peak signal-to-rms-noise improvement figure for a simple 
feedback type comb filter with a pulse train having a (sin Ct/Ct) 
envelope. 


the storage tube filter may be represented by a feedback 
loop having the open loop transfer function e ’e *°”. 
The storage tube accepts input information only during 
the write sweep and supplies output data only during 
the read sweeps. This gating operation can be represented 
by properly timed switches before and after the feedback 
loop, the action of which has been neglected in earlier 
analyses.’”''” The equivalent representation of the storage 
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tube thus obtained is shown in Fig. 9. The results of the 
storage tube analysis using the above representation are 
summarized in Appendix III. 


Output 


tt 


Fig. 9—Equivalent representation of a storage tube comb filter. 
Swl-open during read sweeps; closed during write sweeps. 
Sw2-closed during read sweeps; open during write sweeps. 


The signal power to noise power improvement figures, — 


M,, of the storage tube filter and of the simple feedback 
type filter are plotted in Figs. 10 and 11. The M, figures 
of storage tube filters are generally lower than the /, of 
feedback type filters, but the difference between these 
values does not exceed 1.4 db. The peak signal-to-rms- 
noise improvement, M,, of storage tube filters has been 
plotted in Figs. 12 and 13, along with J, of simple feed- 
back type filters. Storage tube filters give approximately 
1/k — 1 times higher values of 1/7, than the simple feed- 
back type comb filters, assuming one read sweep per k 
sweeps of the storage tube. This increase of the peak 
signal-to-rms-noise improvement figure is caused by the 


decrease of the rms noise output of the filter by the gating | 


action of the storage tube. The peak signal-to-rms-noise 
improvement figure, M,, of the storage tube filter can 


even exceed the /, figure of the optimum filters when k | 


is sufficiently large. However, there is almost no practical 
value in such a storage tube operation since it postulates 
an output device that is able to distinguish a signal peak 
from background noise having a low rms value, but 
high-peak-values in a decreased number of samples. 


V. Discussion oF THE TABULATED SIGNAL-TO- 
Noise IMPROVEMENT IIGURES 


Signal-to-noise improvement figures of the filter types 
described in the two preceding sections are listed in 
Table I. The fundamental assumptions upon which the 
entries are based are as follows: 


1) The rectangular signal pulses have a duration T’,. 

2) The width of the sin «/x shaped pulses is 7, as 
measured between the nulls surrounding the central 
peak. 

3) The idealized low-pass filter preceding the comb 
filter has a sharp cutoff at the frequency w, = 27/T,. 

4) The triangular noise power spectrum decreases to 


one half of its zero frequency value at the frequency _ 


w,. This is equivalent to assuming a predetector 
bandwidth of 2 w,. 


Besides obtaining performance data on specific filters 
within the limitations of the above assumptions, the 
tabulated data help to formulate more general con- 
clusions on the effects of the envelope or pulse shape. 

The sin x/z pulse envelope was introduced to simulate 
the variable amplitude returns of a scanning radar (see 
Section IT). It is seen that a change from rectangular to 
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ig. 10—Signal power to noise power improvement figure of storage 
tube and simple feedback type comb filters k = 4. 
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Fig. 11—Signal power to noise power improvement figure of storage 
tube and simple feedback type comb filters k = 8. 


a sin «/x pulse envelope affects the feeback type filters 
most adversely. Their improvement figures 7, and M, 
are decreased by 4.3 db and 3 db, respectively, if the 
number of pulses between the half-power points of the 
sin 2/x pulse envelope m, is set equal to the number of 
pulses in the rectangular pulse train m. The physically 
realizable optimum filters are affected less under similar 
conditions. Thus, 1/7, is decreased by only 0.9 db, but 
M, is increased by 0.5 db. 

The sin «/x pulse shape of the pulses is intended to 
simulate the effects of pulse distortion. Changing the 
pulses from rectangular to sin x/x shape decreases the 
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Fig. 12—Peak signal-to-rms-noise improvement figure of storage 

tube and simple feedback type comb filters k = 4. 
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Fig. 13—Peak signal-to-rms-noise improvement figure of storage 


tube and simple feedback type comb filters k = 8. 


improvement figures MW, and M, of optimum physically 
realizable filters by less than 2.1 db and 2.7 db, respec- 
tively. The improvement figure 7, of the simple feedback 
type filter is not affected by a change of pulse shape, 
while M, is decreased by approximately 1.3 db. 

A general comparison between physically realizable 
filters and physically nonrealizable filters, which have 
zero transmission stop bands between their pass bands, 
can also be made with the aid of data of Table I. The 
improvement figures M, of both filter types are of the 
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same order of magnitude. Therefore, no general improve- 

ments of filter performance can be expected by increasing 

the attenuation of filter stop bands. Tchebycheff filters of 
1 8.9 . . p . . 

Graves,” which permit an arbitrary degree of attenua- 

tion in their stop bands, appear to have limited merits for 
filtering pulse train signals. 


VI. Dretay TIME INACCURACIES OF ComB FILTERS 


Delay time inaccuracy is one of the factors which 
limits the performance of comb filters, particularly when 
integrating a large number of pulses. In the absence of 
experimental evidence about the structure of delay time 
errors, a normal delay time distribution ‘appears to 
represent a model which will give at least a qualitative 
indication of the expected filter performance. Con- 
sidering cascaded delay line and simple feedback type 
filters, it is assumed that the delay times of the cascaded 
delay line filter are normally distributed about the correct 
delay value, while the delay line in the simple feedback 
type filter is assumed to have normal variations of the 
delay time. The delay time error of a pulse will increase 
with its storage time in either one of the two filters. 
Considering the signal output of the filter at a specified 
time, each pulse of the train has a different probability 
P, of contributing to the filter signal output. The average 
signal output V,, at this time is equal to the sum of all 
the probabilities P; times the signal amplitude S. The 
delay time distribution or variation cause corresponding 
changes of the filter transfer function H(jw). The signal 
output power P,, of the filter is given by an integral 
having H(jw) in the integrand. The average P,, is thus 
the average of an integral. The procedure for computing 
the average values V,, and P,, is described for the 
feedback type filter in Appendix IV. 

When computing the expected signal output of a single 
filter or the average signal output of a set of filters, the 
peak of the average signal V,, occurs at the center of the 
last pulse of the pulse train. The average signal output of 
the cascaded delay line filter during the last pulse of the 
train is plotted in Figs. 14 and 15 for m = 11, and 101, 
respectively. Larger values of the ratio between the 
standard deviation of the delay time and pulse duration 
T,/T, cause the output to decrease in amplitude and to 
spread out in width. For T,/T, = const ¥ 0, the per cent 
drop-off of V,, increases with increasing m. This can 
also be seen from the plot of the filter average peak out- 
puts V,, in Figs. 16 and 17. eit» 

The average signal power output P,, of the two filters 
is plotted in Figs. 18 and 19. P,, of the filters decreases 
less rapidly with decreasing delay time accuracy than 
V.,. Thus, the peak signal output of a feedback type 
filter is decreased by 2 db when the number of pulses m, 
is equal to the ratio of pulse duration to the standard 
deviation of the delay time. The signal power output of 
the same filter is decreased by 1.5 db at the same number 
of pulses, m>. An inaccuracy of the delay time distorts 
the large amplitude output pulses made up of a large 
number of input pulses more than the lower amplitude 
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Fig. 14—Average signal output of a cascaded delay line filter m = 11. 
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Fig. 15—Average signal output of a cascaded delay line filter 


m = 100. 


output pulses. The output signal contributes during 
the buildup and during the decay of the pulse train to the — 
signal power output. These output pulses are less distorted 
than the output pulses which occur during the peak 
signal output. The signal power output is therefore 
affected less by delay time inaccuracies than the peak 
signal output. 

The noise output of the filter is not changed by delay 
time errors which are small compared with the pulse 
repetition period. Therefore, the signal-to-noise improve- 
ment figures of the filters decrease proportionally with — 
the decrease of the average peak signal or of the average 
signal power output of the filter. } 


VII. CorRELATION TECHNIQUES 


Correlation provides a means for improving the signal- 
to-noise ratio of pulse train signals. The correlator per- 
forms in time domain an operation similar to the operation 
of a comb filter in frequency domain. The comb filter 
passes harmonically related frequency components, while 
suppressing bands of frequencies lying between the 
passed frequency components. The correlator multiplies 
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Fig. 16—Average peak signal output of a cascaded delay line filter. 
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Fig. 17—Average peak signal output of a feedback type filter. 


the incoming signal with a reference signal and then 
averages the multiplier output in a low-pass filter. Only 
those frequency components present in both the input 
and the reference signal result in a zero frequency multi- 
plier output component that passes through the low-pass 
filter. Lowering the cutoff frequency of the low-pass 
filter in the correlator is equivalent to narrowing the 
width of the comb filter pass bands. 

The crosscorrelator multiplies the noise-perturbed 
input pulse train with a locally generated reference signal 
which is identical to the input signal with the noise 
removed. The multiplier output is integrated. The auto- 
correlator multiplies the input signal delayed by one 
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Fig. 18—Average signal power output of cascaded delay line filter. 
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Fig. 19—Average signal power output of a simple feedback type 
filter. 


pulse-repetition period with the undelayed input signal. 
The multiplier output is integrated. For minimum noise 
output, the integration starts when the first pulse appears 
in the delay line output and ends when the last pulse is 
applied to the delay line input. The signal-to-noise 
improvement figures of the correlators for pulse train 
signals can be derived by methods similar to the ones 
used for sinusoidal signals.’ The development leading to 
the signal-to-noise improvement figure for the cross- 
correlator is shown in Appendix V. 

For large signal-to-noise ratios the autocorrelator type 
filter gives 3-db lower signal-to-noise ratios than the 
crosscorrelator.”’ For small input signal-to-noise ratios 
the output signal-to-noise ratio is quadratically dependent 
on the input signal-to-noise ratio of the autocorrelator. 
For small 7,; the autocorrelator is inferior to the cross- 


188. F. George, “Time Domain Correlation Detectors vs. Con- 
ventional Frequency Domain Detectors,’’ Naval Res. Lab., Wash- 
ington, D. C., Rep. 4832; May, 1954. 
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correlator, the output of which is always linearly dependent 
on the input signal-to-rms-noise ratio.’?'' 

The peak signal-to-rms-noise improvement, M/,, of 
the crosscorrelator is the same as M, of the optimum 
filter."* The advantage of the simplicity of the correlator 
(which consists of a multiplier and an integrator) is 
offset by the required knowledge of the starting point of 
the received signal. If the starting point of the incoming 
signal is not accurately known, several correlator 
channels each having a different 7 value must be provided. 
This increases the complexity of the equipment. 


VIII. Concuustons 


Comparison of several types of comb filters shows that 
with proper design their performance figures lie within a 
few decibels for a given number of signal pulses in the 
pulse train. The power spectrum of noise and the pulse or 
pulse envelope shapes are only of secondary significance. 
The feedback type comb filter, which is the simplest one to 
construct, gives signal-to-noise improvements within 2.5 
db and 4-6 db of the improvements possible with optimum 
filters for rectangular and sin x/x shaped pulse envelopes, 
respectively. The storage tube filter gives higher peak 
signal-to-rms-noise ratios than the feedback type filter. 
However, it does not appear feasible to make practical 
use of this difference. 

Inaccurate delay times of the filter delay lines affect 
the signal output of the filter. The peak signal output of 
the feedback type filter is decreased by 2 db when the 
number of pulses is equal to the ratio of pulse duration 
to the standard deviation of the delay time. The signal 
power output of the same filter is decreased by 1.5 db 
at the same number of pulses. 

Crosscorrelators give the same peak signal-to-rms- 
noise output as the optimum filters. 
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RITERIA OF FILTER PE 2MANCE 
CRITERIA OF FILTER PERFORMANCE 


The signal power at the filter input is given by 


P= f "| Vie [Pde (2) 
where V{(w) is the signal spectrum, 7’, is an averaging 
interval, w, = 27/T,, and T, is the pulse duration. For 
rectangular pulses this integration interval encompasses 
90 per cent of the total signal power. For sin x/x shaped 
pulses it encompasses all the signal power. Although the 
selection of this interval is arbitrary, it is consistent with 
other investigations.* The noise power input is 


jn / * Paya (3) 


where Py(w) is the power spectrum of noise. The output 
signal and noise powers are 


P= [| vie) P| HGs) Pd) 


—Wp 
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Py =f” | H(Ga) |? Pr) do. (5) 


—Wp 


Expressing the signal-to-noise ratio in decibels, 
yee 
ry) = BLU MOg ae (6) 
and 
oe. ' 
t= Os low Pe (7) 
The power improvement figure of the filter is 


M,; = Too — Tei (Ab): (8) 


The improvement figure is related to the number of pulses 
in the pulse train, m, or to the number of pulses between 
the half power points of the pulse envelope, m,, as 


M, = 10 log 0.45m + f(m) + ¢ (9) 
M, = 10 log m, + c. (10) 


The functions f(m) and constants ¢ depend on the par- 4 


ticular filter type under consideration. The proportionality | 
factor 0.45 in the first term of (9) was selected to provide | 
a direct comparison with the results of George.* 

The peak signal input V,,; is equal to the signal ampli- 
tude S 


Wag — S. 
The peak signal output V,, is given by 


(11) 


Voc max |. Ves) emax 


i 7 WAH Get? de | a 


id —Wp 


The rms noise input and output voltages are given by 


Ey; = V Py: (13) 
and 
Hine = Vv IP (14) 


assuming 1-ohm impedance levels. Expressing the signal- 
to-noise ratios in decibels, 


Vai 
t,, = 20 log Be (15) 
and 
Wes 
iew=20doe fine (16) | 
The voltage improvement figure of the filter is 
M, = Ts. — 1; (db). ake 


The improvement figure is related to the number of 
pulses in the pulse train, m, or to the number of pulses 
between the half power points of the pulse envelope, m,, 
as 


M, = 10 log m+e (18) 


M, = 10 log m +c. (19) 


he constants, c, depend on the particular filter type 
inder consideration. 


APPENDIX IT 


SigNAu-To-Noise IMPROVEMENT FIGURES OF 
THE FEEDBACK Typp FintTEers 


The signal-to-noise improvement figures of the feed- 
ack type filters are computed by substituting the filter 
ransfer function and the signal and noise parameters in 
he relations listed in Appendix I. 

For rectangular pulses of constant amplitude and of 
riangular or rectangular noise spectrum, M, is given by’ 


eee 2A 
aera 


nd M, is given by” 


20 log [aa = Ana 2 A] 


which is plotted in Fig. 7. For rectangular pulses and a 
in Ct/Ct envelope of the pulse train, 


M, = 10 loz | 


Wie 


(21) 


M, = 10 log [s tan” e 2 7 Si (22) 
iM, = 20 log | s.aTa(e ar a aoa) | 
| + 10 logit, (23) 
Lt the time of the peak of signal envelope or 
IM, = 20 log [sateen + ee 1c 0.392) 

+ 10 log 4 (24) 


at the time of the half power point after the peak of the 
pulse envelope, where 


ae 0.5126 — 0.0054 —__—-0.5068 — 0.480 (25) 
8 + 0.6586 + 1.108 6 0.34 

| B= 7/TC (26) 

ee (27) 


A plot of (23) and (24) is shown in Fig. 8. The improve- 
ment figure /, is maximized for a uniform pulse train if 


| y = 125/m (28) 


and for a sin Ct/Ct envelope pulse train at the half power 
point after the envelope peak if 


19 The expression in the square brackets differs from the corres- 
ponding expression listed by Freedman and Margolin, op. cit., by a 
constant factor, 1.17, which is dependent on the low-pass charac- 
teristics. Limiting the interval of integration for computing the peak 
signal output in (12) is equivalent to assuming a rectangular low- 
pass filter with a cutoff frequency #, = 27/7’, in series with the 
comb filter. This low-pass filter gives rise to the factor 1.17 accord- 
ing to footnote 21. 
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y = AT/T, = Alm, (29) 


Substituting these conditions, (20)-(24) can be reduced 
to the form of (9), (10), (18), and (19). The numerical 
values of the constants ¢ are given in Table I. 


AppENDIXx IIT 
SrorAGE Tune Iinrer 


The noise power output in the storage tube represen- 
tation of Fig. 9 will be determined by the gating sequence 
and by the feedback loop characteristics. Following the 
method given by Lawson and Uhlenbeck” it can be shown 
that the gating affects only the amplitude of the noise 
power spectrum without changing its spectral distribution. 
The noise power output of the storage tube filter Py, can 
be related to the noise power input Py; by multiplying 
a factor representing the noise suppression in the feedback 
loop with gating factors. This results in 


k-11=¢" 
aceon’ 


Pye Py; (30) 
assuming one read sweep per k sweeps of the storage tube. 

The signal output of the storage tube will consist of a 
gated pulse train. The output waveform of the filter 
V,.(t) is computed in time domain for an input signal of 
m rectangular pulses of pulse duration T,. 


(m/k)—1 co 
Vie Ra a Gate ees aad 
a=0 a=(m/k) 


‘{1[t- (k-144 qkh)T] 


ie-T,-(k-1+ qT} BD 

where 
e= (Se) Ser eae) (32) 
b=e a (33) 
eee Pos ae ee i | 
A 26 E en 1(m k) i_ oer (34) 


1(a) is a unit step function and ( m/k ) designates the 
largest integer not exceeding (m/k). The peak signal 
output may occur during the output gates prior to or 
after the last pulse of the pulse train. Taking into account 
the low-pass characteristics,* the peak filter output 
during the gate prior to or during the last pulse of the 
pulse train is 


Vi= INSEE SN GT _ e ’)(a = ee SER 


with a and b given by (82) and (33). The peak filter output 
during the gate after the last pulse of the pulse train is 


(36) 


(35) 


Ve; = PASC — eo Agee 
with A given by (34). 


20 Lawson and Unlenbeck, op. cit., pp. 253-254. 
21). A. Guillemin, “(Communication Networks,’ John Wiley and 
Sons, Inc., New York, N. Y., vol. 2, pp. 485-486; 1949. 
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The signal power output is computed as in (4) after 
computing the signal spectrum V{,(w) of the signal 
V,.(t) in (81). This results in 


a = ery a! (2) — 20 


—2(m/k)ky 
sll 


—2ky(m/k) 
+b? —*_____ 4 4? Ba (37) 
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where a, b, and A are given by (82), (33), and (84), re- 
spectively. 
APPENDIX IV 
Errects or INaAccuRATE DELAY TIMES IN THE 
SIMPLE FEEDBACK TypE I'ILTER 


A simple feedback type comb filter with a transfer 


function 
HGw) = (1 + Aei?™)* (38) 


and with inaccurate delay time 7), is considered in this 
appendix. The output of the filter consists of a weighted 
sum of past inputs given by 


Vine) 7 (t) + AV,,(t — T,) + A’V..(E 
+ A™°Y,,[t — (m — 1)T;]. 


= Oy ae. 
(39) 


The delay time of the delay line is assumed to vary 
normally with the mean value equal to 7 and with the 
standard deviation equal to 7’,. 

A pulse recirculating (¢ — 1) times will have the dis- 
tribution 


DT ey NOS SA (40) 


for its deviation from the mean delay time, assuming a 
negligible change in the delay time, 7, during the pulse 
storage. The input of the filter consists of rectangular 
pulses of amplitude S and of duration T,. The time, ¢, 
during the last pulse of the pulse train measured relative 
to the leading edge of this pulse is defined as 


co he a t ta tie (41) 


where 
O< Res il, (42) 


The probability of the 7th pulse to contribute to filter 
output at time ¢ during the last pulse of the pulse train is 


1 2p Sy 5-1) Te)? 
Pe as / 1/2[z/(i-1) Tol? 43 
V2nT (i — 1) J-a-n 75 oe 
where 2 > 2. Introducing the notation 
Pte) = my? fe ae (44) 
0 
(43) becomes 
se eal al ae = | 
a Ee = a cf tle Savant 
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The average output of the filter at the time ¢ defined by# 
(41) is a weighted sum of the probabilities P; plus one, 
multiplied by the ant: of the input signal S 


Be: Lear 
For small values of m, the summations are evaluated 
directly. For larger values of m the summations are 
approximated by integrals. 


Vote 


Assuming inaccurate delay times of the delay line of | 


the comb filter, signal power output will be decreased. 
The signal power output of the filter is 


ae -1 2 
Pes a (2rr) GT) 0.501", 


n=—-Tfp 


where V denotes a fourfold summation of integrals 


SA ie 


ae > Sn de —jwn(i- LENO News 


t=0 k= a=0 p= 
2a (n+1)/T 
—jw(i-k+q—p)T 
| oP a ce 
Znn/T 


w, is defined by 


— Qn + Ir 
oO, = T 


and the probability distribution of the delay time error 
swe 


P(E) =< On) ome 
the signal output power, P,,, is obtained from (47). 


APPENDIX V 


CROSSCORRELATORS 


In the crosscorrelation type filter, the input signal is 
multiplied with a locally generated noiseless reference 
signal, and the multiplier output is then integrated. The 
signal, V,,(¢), and the reference signal, g(t), are both 


considered to be pulse trains of m pulses of duration 


T, and of pulse-repetition period 7. Thus 
m=1 

V.Ad) = S D> [1 — iP) — 1 — 1T — T,)] = Sot). (51) 
i=0 


Denoting the input noise by Vy(¢) an integrator following 
the multiplier will have an output — 


mT 
Vin = [Val + Voll + Dat. 62) 
The maximum signal output which occurs at r = 0 is 
» mT 
V war =) V (0) Sas cre | Vuy(dg) dé (53) 
i) 


assuming undistorted signal pulses. The idealized low- 
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(46) 


(Tfp)-1 2 2 | 
oe (= ObeTs V (47) 


(49) 


(50) | 


After computing the average value of V the average of — 


\) 


iss filter of cutoff frequency w, spreads out the input 
ulses”’ and decreases their area during the undistorted 
ference pulses. This will decrease the signal output 
dicated in (53) by a factor 


a 


l| 


Tp 
(xT,)7? [ (Sie) — Sia. er lat 


l| 


20 
r | Sivde 0.9. (54) 
0 
‘he autocorrelation function of the noise after the multi- 
ee with the pulse train, g(t), is the product of the 
spective autocorrelation functions 


DPHO MT =n. 
Ri) = . ) T T sin WT (55) 
tric) < 7, 
B(r) = 0 (56) 


r |r| > T,. The mean square noise output of the filter 
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corresponding to the instantaneous output of (53) is 
obtained as” 


Pose i Gul =e ances (57) 
0 


Substituting R,(7) of (55) and (56) in (57), evaluating 
the integral, and assuming 7’, much less than T 
Pyo = 4Py(0)T,mSi(2r). (58) 


Using the signal output of (53) in conjunction with (54), 
the peak signal-to-rms-noise improvement figure of the 
crosscorrelator becomes 


Voh/ae 
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Non-Mean-Square Error Criteria’ 


SEYMOUR SHERMANT 


_ Summary—While in the engineering literature non-mean- 
'quare error criteria for predictors are often presented as physically 
ignificant and then shunted aside because of mathematical un- 
| Eee it is shown here that in the case of Gaussian pro- 
esses all such criteria given in three recent textbooks yield the 
ame predictor as the linear minimum mean-square predictor of 
Wiener. 


N the usual engineering presentation’ of the 
Wiener theory of prediction, it has become con- 
ventional to make remarks relating to 1) Gaussian 

tatistics, 2) linearity of the predictor, and 3) the mean- 

quare error criterion. It is generally remarked* that 
vhile other error criteria are “physically appropriate,” 
hey are “apparently too difficult to handle mathemati- 
ally” and the mean-square error “lends itself conveniently 


* Manuscript received by the PGIT, April 24, 1958. 

+ Moore School of Elec. Eng., University of Pennsylvania, 
Philadelphia, Pa. 

LJ. G. Truxal, ““Automatic Feedback Control System Synthesis,”’ 
McGraw-Hill Book Co., Inc., New York, N. Y., pp. 410-499; 1955. 

2 J. H. Laning, Jr. and R. H. Battin, “Random Processes in Auto- 
natic Control,’”? McGraw-Hill Book Co., Inc., New York, N. Y., 
yp. 172-182; 1956. 

3W. B. Davenport, Jr. and W. L. Root, “An Introduction to 
he Theory of Random Signals and Noise,’ McGraw-Hill Book Co., 
ne., New York, N. Y., pp. 219-221; 1958. 

4 Laning and Battin, op. cit., p. 180. 


to mathematical analysis.” With these regrets, the con- 
sequences of the mean-square error criterion become the 
limited subject of the investigation. Frequently it is 
remarked correctly’ that in the presence of Gaussian 
statistics the mean square criterion yields an optimum 
predictor which is linear, so that no nonlinear predictor 
is better than the Wiener predictor in the case of Gaussian 
statistics and the mean-square error criterion. The purpose 
of this paper is to point out that the rejected error criteria 
are mostly of the form H¢(e), where e is a random variable, 
the error, while ¢ satisfies 


0 < ¢@ = ¢(-2—; 0S$aq 5a >¢@) S $@) (1) 


and in the presence of Gaussian statistics such an error 
criterion yields a linear predictor as the optimum pre- 
dictor. This is the same linear predictor which is yielded 
by the mean-square error criterion. Thus, in the presence 
of Gaussian statistics, the results of the Wiener mean- 
square error theory of prediction, either operating on 
data from the infinite past or only the finite past, have a 
wider applicability than is usually realized; they apply 
automatically to almost all of the error criteria that have 


5 Davenport and Root, op. cit., p. 231. 
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been proposed. The same integral equation yields the 
solution of all these cases. 

The first lemma is well known.” 

Lemma: If ¢ satisfies (1) and F is a probability dis- 
tribution function on the reals which is symmetric and 
unimodal with mode at the origin so that F(X) = 1 — 
F(—X) at each continuity point of F and F is convex for 
x < 0, then 


7 


[eco arcx) s | 9X - @ ax) 


for each real a, when the integrals exist; if either integral 
diverges, the one on the right does. 

It follows in particular that if the distribution function 
is absolutely continuous, then the lemma holds. We use 
only this version, and the reader can convince himself 
of its truth by drawing a diagram. 

Now consider a Gaussian process {s,; —-7 <t <a} 
and the problem of finding the estimate s,, f > 0 in terms 
of {s,: t S 0}, which minimizes H¢(e), where 


e = 8s, — &,({s,: ¢ S 0}) (2) 
and ¢ satisfies (1). Note that 
Ege) = L[E@O)]|{s.: ¢ S 0})]. (3) 


Since the conditional distribution of s, on the hypothesis 
{s:: t S 0} is Gaussian and therefore unimodal and 
symmetric about the mode, H(s,|{s,: t S O}), we see 
that §, = E(s,|{s,: £ S 0}) minimizes H(¢(e)|{s,: t S O}) 
and therefore minimizes H[E(¢(e)|{s.: ¢ S O})] = Hole). 
Thus, in the case of a Gaussian process, 


0}), 


which is the solution of the Wiener integral equation and 
is that predictor which minimizes E(e’), also automatically 
minimizes E(o(e)) where @ satisfies (1). 

Among the ¢ satisfying (1) which have been offered in 
the literature and then rejected on account of math- 
ematical complication are 


8, = E(s; | EY t < 


6 T. W. Anderson, ‘The integral of a symmetric unimodal func- 
tion over a symmetric convex set and some probability inequalities,”’ 
Proc. Amer. Math. Soc., vol. 6, pp. 170-176; 1955. 

78. Sherman, ‘‘A theorem of convex set with applications,” 
An. Amer. Math. Stat., vol. 26, pp. 763-767; 1955. 
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gle) = |e | Pee 

Ppa eriel 
ly, Roeace 7 
0,- erhe<ca, 

40 95) = ee 
dec oe dp 1ebst een ede 


It should be further pointed out that considerations — 
similar to those presented here apply to the case where | 
- > f, > 018m 
, €,) is minimized. Particularly 
in the case of a Gaussian process, the two criteria (5.1-5) | 
and (5.1-6) of Laning and Battin* yield the same optimum 


prediction for several times f; > fs > 
carried so that H¢(e, --- 


predictor. 


In the case of a non-Gaussian process, where the — 
conditional distribution of s, given {s,: ¢ S 0} satisfies — 


the conditions of the lemma, again the optimum predictor 


in the sense of minimizing H¢(e), where ¢ satisfies (1), | 


is the same as the minimum mean-square error predictor, 


although there is no guarantee in this case that the | 


predictor is linear. 


It is a pleasure to acknowledge the stimulation of — 


students in my course EHE668, Engineering Statistics 


Seminar. On one occasion last semester I pointed out the © 
well-known fact that the optimum predictor for least-— 
square error criterion was conditional expectation’” and — 
that for Gaussian processes the conditional expectation | 
was linear.’ At this point, in response to a question by | 
I. Maron, N. C. Randall indicated that for the absolute | 
deviation as error criterion, as given by Cramer” the | 


conditional median should be the optimum predictor. 
This exchange led the author to observe that condition 


(1) leads to a collection of manageable error criteria (for | 
Gaussian processes) which included not only ¢:, but the | 


previously rejected ¢, and ¢3. 


5 Laning and Battin, op. cit., pp. 179-180. 

® Davenport and Root, op. cit., p. 220. 

10 Truxal, op. cit., p. 413. 

4 [bid., p. 414. 

2H. Cramer, “Mathematical Methods of Statistics,’’? Princeton 
University Press, Princeton, N. J., pp. 271-272; 1946. 
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4. Comment on Pattern Redundancy* 


_ Arecent paper by Glovazky! discussed a 
ethod for determination of redundancies 
in a given, finite set of P two-tone patterns 
such that each pattern could be described 
oy a finite number C of cells, each all white 
pr all black. Glovazky draws up a code 
schedule, consisting of the numbers 0 
white) and 1 (black) to describe the 
scanning process and effect separation. 
Glovazky has shown that at most P — 1 
bells are necessary to effect separation. The 
‘east possible number is the smallest integer 
y satisfying y > logeP. One problem is the 
selection of the smallest number z of cells, 
ogeP < z < P — 1, that makes separation 
possible. Call these ‘“‘special’’ cells. 
Consider first the case P = 2”. If we can 
find y special cells out of the C available 
ells, such that the reduced “‘code’”’ schedule 
or these cells, arranged in descending value, 
ee all binary numbers from P — 1 to 
)), then the minimum separation path has 
heen found. An aid to finding these special 
cells is the following necessary condition: 
The number of ‘“ones’’ (also called ‘““weight’’) 
pn each cell column for these y cells is exactly 
/ Fa/ 2, 
In the more general case, the following 
recone condition for finding y separation 
gells can be used to aid in the selection 
f these special cells. Write out in ascend- 
ng order the binary numbers from 0 to 
{2u — 1). Bracket the numbers from 0 to 
— 1 inclusive, and count the number of 
‘ones”’ in each bracketed column; denote 
these by mo, 71, M2 +--+ my_1. Now bracket 
tthe numbers from (2¥ — 1) to (2¥ — P) in- 
lusive, and count the number of “ones” 
n each bracketed column; denote these by 
0) M1, M2 *** Ny. Obviously, m; +n; = P. 
Then a necessary condition for finding y 
ells Co, Ci +++ Cy_1 to effect Separation is 
that cell column C; contain Cj “ones,’’ where 
ih SSG SS OR This process is ESR in 
1 i il; 
Consider Glovazky’s example, with P =6, 
= 9. In this case, we find mp = 3, m = 2, 
ms = 2, hence 3 < co < 3 (see Fig. 1). 


1 


¥ 


2<a <4 
250<4, 


In Glovazky’s code schedule, cell columns 
3, 5, and 9 each contain 3 “ones.’? Exami- 
ation indicates that these three cells effect 
eparation, and it is not necessary to use 
our cells. 

Consider next Glovazky’s alpha-numeric 
ase with C = 100, P = 35. Here y = 6, 
(Pp — 1) = 34. Using Glovazky’s method, 
we would find 34 or less cells to effect sepa- 
ration. To find 6 special cells—using trial 
and error—could prove a formidable task, 
since one can choose 6 cells out of 100 in 
about 10° ways. Using the necessary con- 


* Received by the PGIT, October 2, 1957. 

1A, Glovazky, ‘‘Determination of redundancies 
in a set of patterns,’’ IRE Trans. on INFORMATION 
Tuerory, vol. IT-2, pp. 151-153; December, 1956. 
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(a) Bracketed binary number sequence. 
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(b) Finding m,;. 
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(c) Finding ,;. 


Bigs. 


dition above, we find 
17 Co S18 
NSCS 
19 
19 
19 
3S S 3 


eS eS 
(or) aS Oo 

IA IA IA lA 
croae 

VA WA WA WA 


Hence trial should start with those cell 
columns containing 17 or 18 “‘ones.’’ Con- 
versely, if there is no column containing 17 
or 18 “ones,”’ then separation with six cells 
is not possible. 


O. LowENscHuss 
Sperry Gyroscope Co. 
Great Neck, N. Y. 


Relative Efficiency of English and 
German Languages for Communi- 
cation of Semantic Content* 


Since the formulation of the statistical 
theory of communication!” nearly a decade 
ago, interest in the statistical properties of 


* Received by the PGIT, December, 1957. 
1C, Shannon, “A mathematical theory of 
communication,” Bell Sys. Tech. J., vol. 27, pp. 379- 
423; July, 1948. 
2'N. Wiener, ‘Extrapolation, Interpolation and 
Smoothing of Stationary Tinsel mite pe sonn Wiley 
and Sons, Inc., New York, N. Y.; 
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language, the chief vehicle of communi- 
cation, has been greatly stimulated for 
many practical reasons, such as the con- 
struction of efficient codes. The view that 
language itself may be regarded as a code 
for certain conceptual entities has been fre- 
quently emphasized in current linguistic 
discussions. This is largely due to the fact 
that Shannon’s measure for the entropy of 
a stochastic process has been found to be 
readily applicable for the calculation of the 
selective information content of language 
considered as a set of discrete code symbols. 

This view carries with it, however, certain 
implications which do not seem to have 
been explicitly stated or exploited. For, if 
we regard, by*way of an illustration, the 
two statements, “All that glitters is not 
gold”? in English and its translation, “Es 
ist nicht alles Gold was glanzt”’ in German 
as alternative codes having the same mean- 
ing or significance, it is immediately evident 
that a comparison between the two lan- 
guages is possible with regard to the ef- 
ficiency with which they encode the same 
semantic content? (into linguistic symbols) 
without making reference to its absolute 
measure. To use a metaphor translation 
from one language into another may be 
regarded as a transformation of the code 
which leaves the semantic content, but not 
the selective entropy of the code symbols, 
invariant. To be sure, the transformation 
is not unique—and this is significant—but 
comparison of different languages, con- 
sidered as alternative codes,is still possible 
at a statistical level, and leads to interesting 
results. For instance, one may take a certain 
sample of text in English and also its trans- 
lation, say, sentence by sentence, in Ger- 
man, and determine the number of bits of 
information represented by the original 
English text, Ng, and its translation in 
German, Ng, computed on the basis of, say, 
the letter frequencies obtained in the origi- 
nal and in the translation. Nz and Ne are 
the figures of interest if a decision is to be 
reached on the question whether the mes- 
sage should be sent in English or in German 
using optimal binary codes based on letter 
frequencies in these languages. More gene- 
rally, by considering sufficiently long and 
semantically equivalent statements in dif- 
ferent languages we can compare the rela- 
tive communication efficiencies of the 
languages in several respects such as their 
efficiencies for transcription if their orthog- 
raphies are different. A comparative study 
of some of the major languages of India on 
these lines is in progress here and is reported 
elsewhere.* 

We shall confine ourselves here to a com- 
parison of the English and German lan- 
guages considered as alternate codes which 
carry the same semantic content. With this 
object, a number of articles were taken from 


3 The phrase “‘semantic content” is used here as 
more or less synonymous with “‘meaning’’ as currently 
ued in the literature on information ‘theory. 

S. Ramakrishna, et al., ‘Statistical studies in 
ee Indian languages with applications to com- 
munication engineering,” J. Inst. Telecomm. Engrs. 
(India), vol. 4, pp. 25-35; December, 1957. 
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recent numbers of The Reader's Digest for 
which sentence by sentence translations 
were readily available in its German edition. 
It is, of course, possible to obtain a rough 
estimate of the relative coding efficiencies 
of the two languages by merely counting 
the total number of letters ny in the English 
text and ng in the corresponding German 
text, and multiplying them, respectively, 
by the entropies Hy and Hg known on the 
basis of letter frequencies in these lan- 
guages. In view of the rather short size of 
the sample (about 35,000 letters), it was 
decided to compute the letter frequencies 
actually obtained in the original and in the 
translation and calculate the entropies per 
letter appropriate to the texts under con- 
sideration. 


TABLE I* 
Ger- | Ger- 
English man «| man_ |IEnglish 
\(original)| (trans- | (origi- | (trans- 
lation) nal) Jation) 
Number of letters 
in sample, n 33,980 | 44,376} 36,466] 32,821 
Number of word 
spaces in sample, 
s 7900 6579 6536 7381 
Total of letters 
and spaces, n + s| 41,880} 50,955} 43,002] 40,202 
Entropy disregard-| 
ing word space, 7; 4.170 4.080} 4.086) 4.150 
Entropy counting 
word space, H’ 4.083 4.074 4.081; 4.079 
Total entropy 
without word 
space, n X H 141,697 | 181,054) 149,000} 136,207 
Total entropy 
with word space, 
(n + s) H’ | 171,017 | 207,590) 175,491) 163,984 
i | 


*Number of letters, words, entropies per letter, 
etc., of samples of English and German texts which 
are translations of each other and hence are seman- 
tically equivalent. The second and third columns 
give data relating* to translation from English to 
German and the last two from German to English 
translation. 


In the second and third columns of Table 
T are given in order the number of letters n, 
the number of word spaces s, their sum 
n + s, the entropy H disregarding the word 
space, the entropy H’ counting the word 
space as a symbol, and the total number of 
bits of information n X H and (n + s) H’ 
in these two cases for the English (original) 
and the German (translation) texts. Naive 
comparison at this stage shows that if we 
ignore the word space (next to last row), 
in German we require 181,054 bits as against 
the 141,697 bits required in English to ex- 
press the same semantic content. It appears 
that about 30 per cent more bits of infor- 
mation are required in German than in 
English, or to put it more concisely, the 
semantic value of a “German-bit”’ is about 
0.78 of an ‘“English-bit.’? The comparison, 
however is unfair to German in two re- 
spects. First, it is well known that the word 
space has a significance in communication 
and is not altogether irrelevant. If we, 
therefore, take the word space into account 
we notice from the last row that German 
fares better than before, a German-bit being 
equal to 0.82 of an English-bit. Second, 
there is the possibility that the fact that 
the original text was in English could have 
given an advantage to English. This may 
be argued on the basis that when one does 
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ARTICLES FROM THE READER’S Digest WHIcH WERE USED IN 
THE ENGLISH TO GERMAN TRANSLATION 


English (original) 


[1] ‘‘ ‘Hot’? Atoms—Industry’s Versatile 
Detectives,” (pp. 87-92, April, 1957). 

[2] “What Makes a Woman Memorable,” 
(pp. 35-36; April, 1957). 

[3] “Nightmare on the 79th Floor,’’ (pp. 
37-41; April, 1957). 

[4] “Coping with the Compliment,” (pp. 
42-44; April, 1957). 

[5] “They’re Shipping Freight by Pipeline 
Now,” (pp. 57-60; April, 1957). 

[6] ‘Discovering the People of Paris,’’ (pp. 
61-62; April, 1957). 

[7] ‘““Acne—Youth’s Mysterious Enemy, 
(pp. 78-80; April, 1957). 

(8] “Work!” (pp. 85-86; April, 1957). 


”? 


German (translation) 
“ “Heisse’ Atome,”’ (pp. 69-72; April, 1957). 


“Was eine Frau interessant macht,” (pp. 
35-36; May, 1957). 

“Inferno im 79. Stock,’ (pp. 57-62; May, 
1957). 

“Wie man mit Komplimenten fertig wird,” 
(pp. 132-140; May, 1957). 

“Pipelines haben Hochbetrieb,” (pp. 68-72; 
May, 1957). 

“Unter den Menschen von Paris,” (pp. 73- 
75; May, 1957). 

‘“‘Akne—Kummer der Jugend,”’ (pp. 48-51; 
May, 1957). 

““Arbeite,”’ (pp. 84-86; May, 1957). 


ARTICLES USED IN THE GERMAN TO ENGLISH TRANSLATION 


German (original) 


{1] Das Leiden eines Knaben, (Bilingual 
Series), Conrad Ferdinand Meyer, George 
G. Harrap & Co., Ltd., London, Eng., 
pp. 4-10, 54-64; 1949. 

[2] Deutscher Stahlbau sein Feld ist die Welt, 
Deutscher Stahlbau-Verband, Koln, 
Ger., “Die Welt, die Stahlbauingenieure 
sehn,”’ pp. 12-19. 

[3] Chemische Grundbegriffe, Alfred Benrath, 
George G. Harrap & Co., Ltd., London, 
Eng., ‘““Wesen und Wege der Naturfor- 
schung,”’ pp. 4-10. 


translation, there is a certain freedom of 
choice among alternative expressions pos- 
sible in the language into which translation 
is made and that this uncertainty which 
is equivalent to noise! (in the sense of the 
information theory) must be overcome by 
the use of additional bits of information. 
To check the validity of this argument 
we considered samples of texts with the 
original in German and the translation in 
English, the material being drawn from 
various sources. The corresponding figures 
for the number of letters, the entropies, etc., 
for German and English are given in the 
last two columns of the same table. We 
notice from the last two rows of these 
columns that a different ratio obtains be- 
tween a German and an English-bit, and 
that, in particular, taking the word space 
as a symbol (last row) one German-bit is 
equal to about 0.94 of an English-bit. A 
bit in German even now carries somewhat 
less semantic content than a bit in English, 
but the advantage now rests with German. 
In this situation we naturally ask whether 
it is possible to find a more exact relation 
between the semantic value of an English- 
bit and a German-bit. If we can assume 
that the extent of uncertainty in translation 
from German to English is the same as that 
from English to German then this relation- 
ship can be readily established. (Such an 
assumption appears reasonable in the case 


English (translation) 


The Tribulations of a Boy, translated by 
E. M. Huggard. 


German Steel Construction, Its Field Is the 
World, ‘The world that civil engineers see.” 


The Fundamental Ideas of Chemistry, trans- 
lated by Jethro Bithell, ““The nature and 
the methods of scientific research.” 
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of two closely related languages like English 
and German though it may not hold good 
for widely different languages.) We merely 
have to make allowance for the additional 
number of bits required to overcome the 
‘noise’ in the total number of bits required 
in the language of translation. Making this 
assumption, let one bit of English be equal 
to x bits of German and let y bits be the 
noise accompanying each bit of information. 
Considering the word space as equivalent 
to a letter symbol, we can form from the 
last row the two equations 


171,017 (2 + y) = 207,590 
175,491 (a! + y) = 163,984 


for the English to German and the German 
to English translations. Solving for x and y 
we obtain 


a = 1.149 and y = 0.065. 


From the small samples studied, it thus 
appears that the English language is some- 
what more efficient than the German lan- 
guage in encoding semantic content into 
linguistic symbols to the extent that one 
bit of information in English carries ag 
much semantic content as 1.15 bits of in- 
formation in German. It appears also that 
the process of translation is generally ac- 
companied by some degree of uncertainty 
and to overcome this one requires additional 
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bits of information. The authors wish to 
point out the desirability of using larger 
samples of more varied character to obtain 
more precise figures. The tedious nature of 
the task of counting the different letters 
(in the absence of mechanical aids), how- 
ever, limited the authors to the present 
size of the sample. 


B.S. RAMAKRISHNA 
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The Optimal Distribution of Signal 
Power in a Transmission Link 
Whose Attenuation Is a Function of 
Frequency* 


When a transmission line having attenu- 
ation and noise is used as an element of a 
‘communication channel, the channel capac- 
‘ity depends on the nature of the signal. If 
‘the transmission line is linear and the noise 
iis Gaussian noise, a formula for the channel 
(capacity can be written explicitly. If the 
|total signal power is limited, there is an 
upper bound to the channel capacity, and 
the maximum channel capacity can be ap- 
| proached only with signals having a certain 
' frequency spectrum. 


| In practical cases, the frequency spec- 


trum of the signal may be dictated by 
practical requirements other than informa- 
tion theory. For example, a flat band- 
limited spectrum may be desirable. 

Furthermore, the effective channel capac- 
ity may be limited not by the theoretical 
upper bound, but by the signal-to-noise 
ratio in the worst part of the frequency 
spectrum. This is true, for example, in 
frequency multiplex systems, where a multi- 
plex group is good if every channel is quiet 
enough to use, but hardly any better if 
‘some channels are much quieter than the 
minimum standard. 

It is interesting to see in a simple example 
that the effective channel capacity is not 
much less with these restrictions on the 
‘signal spectrum than when the signal spec- 
trum has its optimum shape. If the noise 
spectrum is flat with frequency, the at- 
tenuation is proportional to the square 
root of frequency, and the signal-to-noise 
ratio large, then the channel capacity using 
a flat signal spectrum has asymptotically 
the same upper bound as the channel capac- 
ity with the optimal signal spectrum. If 
‘the effective channel capacity is calculated 
on the assumption that the effective signal- 
to-noise ratio is the signal-to-noise ratio in 
the noisiest part of the band, then the upper 
bound of the effective channel capacity is 
lower by a factor 4/9 than in the optimal 
case. 


* Received by the PGIT, October 1, 1957. 
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Let us suppose that the transmission link 
in question has an input and an output. 
Let us suppose that the attenuation suf- 
fered by a signal of frequency f in passing 
from the input to the output is a(f). Let us 
suppose that the noise emerging from the 
output is Gaussian noise with a power 
spectrum N(f). Suppose finally that a 
signal is inserted into the input which has 
a power spectrum P(f). 

At the output, the signal power spectrum 


Pa ees fe 7 (1) 


The upper limit of the channel capacity? is 


iP log (1 au me) df. (2) 


The total signal power is 


C= 


Ce aaie: (3) 


Following Shannon,? it is clear that the 
maximum of channel capacity for constant 
P occurs when 


RG) iN ee 


if this expression 
is greater than 
Zer0, 


I 


(4) 


= 0 


otherwise, 
where & is a constant chosen to give the 


desired value of total signal power P. 
Now, to be specific, let 


NG) = 
ayes 


(5) 
(6) 


N = constant; 


V$/4fo- 


1C. E. Shannon, ‘‘Communication in the presence 
of noise,’ Proc. IRE, vol. 37, pp. 10-21; January, 
1949. 

2 Tbid., or see E. Goursat, ‘‘Cours d’Analyse,” 
Paris, France, 2nd ed., vol. 3, pp. 575-576; 1915. 


This attenuation law is representative of 
the high-frequency behavior of any trans- 
mission line carrying a TEM wave in which 
resistance is controlled by skin effect, e.g., 
twisted pairs, coaxial lines, and so forth. 
The quantity f, is simply a factor to normal- 
ize the unit of frequency. It is the frequency 
at which transmission through the link is 
one neper more attenuated than at zero 
frequency, 7.e., the ‘one-neper fall-off fre- 
quency.” 

In this case the signal power spectrum is 


Nev tie ee 


ifecake of 


EG) =a 
= 0 


where B is described in terms of a new 
parameter w as follows: 


[log* ( + 1)]/4e*(1), (8) 
(k — N)/N. (9) 


The parameter wu is defined as the signal-to- 
noise ratio at zero frequency. The integrals 
for total signal power P and the upper 
limit of channel capacity C can be evaluated 
to give 


B 


I 


U 


P/Nfp = Cu + 1) log’ (wu + 1) 


— 2u + 1) log w+ 1) + 2u (10) 
C/fo = [log* (w+ 11/8. (1) 


Although it is impractical to solve directly 
for C as a function of P, it is quite feasible 
to represent P/Nf, and C/f, parametrically 
in terms of wu. The results are plotted in 
Fig. 1 to show how C varies with P. 

Now consider another transmission sys- 
tem where the signal power is distributed 
uniformly over a transmission band of finite 
width. As before, the signal-to-noise ratio 
at the receiving end of the link is a function 
of frequency, and we must distinguish two 
cases: the “‘ideal’’ case, where the coding 
takes advantage of the frequency depend- 
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ence of S/N; and the “practical” case 
where the S/N at the top of the transmis- 
sion band (or at the noisiest frequency) is 
accepted as the working S/N for the whole 
transmission band. 

In both of these cases 

PO) ee (12) 

where B is the bandwidth of the channel. 
It will be seen later that the optimum band- 
width B is different in the two cases. 

In the “ideal’’ case 


Cox [ log [1 + (P/BN)e Y"} df. 


Jo (13) 


If we assume that the signal-to-noise 
ratio is large, the integral can be estimated 
by ignoring the unity in the argument of 
the logarithm. If we introduce again the 
zero frequency signal-to-noise ratio 


1 =P) BN. (14) 
we find that C has a maximum when 
B = flog u — 1)’. (15) 


For this value of wu, the values of normalized 
channel capacity and normalized power are 
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C/fo ~~ t[log u — 1]’flog wu + 2], (16) 
P/Nf,p = ullog u — 1)’. Gli) 


At the upper edge of the band 


S/N = (P/BN)e¥?" =e, (18) 


7.e., no matter what the signal power, the 
bandwidth is such that the S/N is one neper 
at the highest frequency in the pass band. 
Of course, this estimate is not exact. 

In the “practical’’ case, we assume that 
the effective S/N for the whole band is the 
S/N at the highest frequency, 7.e., 


S/N = (P/BN)e¥?/", (19) 


In this case the effective upper limit of 


channel capacity is 
Oe i log [1 + (P/BN)e~¥®"] df. 
re) (20) 


To the same degree of approximation’ as 
before 


B 


B = (4/9)(log u — 1)” (21) 
C/fo ~~ (4/27) (log u — 1)” 
‘(log u + 1) (22) 
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P/Nfo = (4/9)u(log u — 1)”. (23) 
Also, at the upper edge of the band, the 
signal-to-noise ratio is 


1/3 2/3 
Uk Cin: 


S/N = (24) 

Fig. 1 shows the normalized upper bound 
of channel capacity C/fo plotted as a func- 
tion of normalized signal power P/Nfp. 
The “optimum” system (shaped signal 
spectrum) is better than the other two, but 
not much: for large signal power, the 


‘Gdeal’’ system (flat signal spectrum) is y 


nearly as good, and the “practical’’ system 
(effective S/N equals S/N at the highest 
frequency transmitted) is worse only by a 
factor 4/9. 

These examples suggest that the upper 
bound of the effective channel capacity of 
a transmission link may not be much af- 
fected by certain constraints on the signal 
spectrum. Hence, we should not hope to 
make big improvements in transmission 
system efficiency by altering signal spectra 
to increase channel capacity. Of course, 
each case must rest on its own merits. 
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